[2026-07-02 06:18:50] Starting L2T batch32 Qwen3-4B GRPO/RLSD_TR dataset-first queue [2026-07-02 06:18:50] Commit: ee6a667 [2026-07-02 06:18:50] Settings: model=Qwen/Qwen3-4B steps=100 train_batch=32 rlsd_tr_batch=32 rollout=8 mini_batch=8 tr_mix=0.1 max_model_len=10240 vllm=0.8 [2026-07-02 06:18:50] Skipping chemistry grpo; completed by previous queue [2026-07-02 06:18:50] Starting chemistry rlsd_tr Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B 2026-07-02 06:18:52,310 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 06:19:01,985 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2026458) TaskRunner hostname: mole-workspace-rw, PID: 2026458 (TaskRunner pid=2026458) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'calculate_entropy': False, (TaskRunner pid=2026458) 'calculate_sum_pi_squared': False, (TaskRunner pid=2026458) 'checkpoint': {'_target_': (TaskRunner pid=2026458) 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra']}, (TaskRunner pid=2026458) 'clip_ratio': 0.2, (TaskRunner pid=2026458) 'clip_ratio_c': 3.0, (TaskRunner pid=2026458) 'clip_ratio_high': 0.28, (TaskRunner pid=2026458) 'clip_ratio_low': 0.2, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_coeff': 0, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'freeze_vision_tower': False, (TaskRunner pid=2026458) 'fsdp_config': (TaskRunner pid=2026458) {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'kl_loss_coef': 0.001, (TaskRunner pid=2026458) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'loss_scale_factor': None, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-06, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': 10, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2026458) 'clip_cov_lb': 1.0, (TaskRunner pid=2026458) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'clip_cov_ub': 5.0, (TaskRunner pid=2026458) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'loss_mode': 'rlsd', (TaskRunner pid=2026458) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2026458) 'alpha': 0.5, (TaskRunner pid=2026458) 'distillation_add_tail': True, (TaskRunner pid=2026458) 'distillation_topk': 100, (TaskRunner pid=2026458) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2026458) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2026458) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'loss_weight': 0.0, (TaskRunner pid=2026458) 'mask': 'all'}, (TaskRunner pid=2026458) 'feedback_template': '\n' (TaskRunner pid=2026458) 'The ' (TaskRunner pid=2026458) 'following ' (TaskRunner pid=2026458) 'is ' (TaskRunner pid=2026458) 'feedback ' (TaskRunner pid=2026458) 'from ' (TaskRunner pid=2026458) 'your ' (TaskRunner pid=2026458) 'unsuccessful ' (TaskRunner pid=2026458) 'earlier ' (TaskRunner pid=2026458) 'attempt:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{feedback_raw}', (TaskRunner pid=2026458) 'full_logit_distillation': True, (TaskRunner pid=2026458) 'include_environment_feedback': True, (TaskRunner pid=2026458) 'is_clip': 2, (TaskRunner pid=2026458) 'max_reprompt_len': 10240, (TaskRunner pid=2026458) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2026458) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) 'Correctly ' (TaskRunner pid=2026458) 'solve ' (TaskRunner pid=2026458) 'the ' (TaskRunner pid=2026458) 'original ' (TaskRunner pid=2026458) 'question.', (TaskRunner pid=2026458) 'reprompt_truncation': 'right', (TaskRunner pid=2026458) 'solution_template': '\n' (TaskRunner pid=2026458) 'Correct ' (TaskRunner pid=2026458) 'solution:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{successful_previous_attempt}', (TaskRunner pid=2026458) 'srpo_dynamic_weighting': False, (TaskRunner pid=2026458) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2026458) 'success_reward_threshold': 0.5, (TaskRunner pid=2026458) 'teacher_regularization': 'trust-region', (TaskRunner pid=2026458) 'teacher_update_rate': 0.1, (TaskRunner pid=2026458) 'token_reweight_decay_steps': 0, (TaskRunner pid=2026458) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2026458) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2026458) 'tau_neg': 1.05, (TaskRunner pid=2026458) 'tau_pos': 1.0, (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_kl_loss': False, (TaskRunner pid=2026458) 'use_prefix_grouper': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'hybrid_engine': True, (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2026458) 'custom_chat_template': None, (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'exclude_modules': None, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2026458) 'hf_config_path': None, (TaskRunner pid=2026458) 'lora_adapter_path': None, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2026458) 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': None, (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_liger': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'nccl_timeout': 600, (TaskRunner pid=2026458) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': True, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2026458) 'agent_loop_config_path': None, (TaskRunner pid=2026458) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2026458) 'name': None, (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2026458) 'num_workers': 8}, (TaskRunner pid=2026458) 'calculate_log_probs': True, (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enable_rollout_routing_replay': False, (TaskRunner pid=2026458) 'enforce_eager': False, (TaskRunner pid=2026458) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2026458) 'ignore_eos': False, (TaskRunner pid=2026458) 'layered_summon': False, (TaskRunner pid=2026458) 'load_format': 'dummy', (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'max_num_batched_tokens': 10240, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'mode': 'async', (TaskRunner pid=2026458) 'multi_stage_wake_up': False, (TaskRunner pid=2026458) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'format': 'hermes', (TaskRunner pid=2026458) 'interaction_config_path': None, (TaskRunner pid=2026458) 'max_assistant_turns': None, (TaskRunner pid=2026458) 'max_parallel_calls': 1, (TaskRunner pid=2026458) 'max_tool_response_length': 256, (TaskRunner pid=2026458) 'max_user_turns': None, (TaskRunner pid=2026458) 'num_repeat_rollouts': None, (TaskRunner pid=2026458) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2026458) 'use_inference_chat_template': False}, (TaskRunner pid=2026458) 'n': 8, (TaskRunner pid=2026458) 'name': 'vllm', (TaskRunner pid=2026458) 'over_sample_rate': 0, (TaskRunner pid=2026458) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2026458) 'port': 9090, (TaskRunner pid=2026458) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'quantization': None, (TaskRunner pid=2026458) 'quantization_config_file': None, (TaskRunner pid=2026458) 'response_length': 8192, (TaskRunner pid=2026458) 'scheduling_policy': 'fcfs', (TaskRunner pid=2026458) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2026458) 'skip_rollout': False, (TaskRunner pid=2026458) 'skip_tokenizer_init': True, (TaskRunner pid=2026458) 'temperature': 1.0, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 1.0, (TaskRunner pid=2026458) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2026458) 'backend': None, (TaskRunner pid=2026458) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2026458) 'token2text': False}, (TaskRunner pid=2026458) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2026458) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'n': 16, (TaskRunner pid=2026458) 'temperature': 0.6, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 0.95}}}, (TaskRunner pid=2026458) 'algorithm': (TaskRunner pid=2026458) {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2026458) 'adv_estimator': 'grpo', (TaskRunner pid=2026458) 'gamma': 1.0, (TaskRunner pid=2026458) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2026458) 'horizon': 10000, (TaskRunner pid=2026458) 'kl_coef': 0.001, (TaskRunner pid=2026458) 'target_kl': 0.1, (TaskRunner pid=2026458) 'type': 'fixed'}, (TaskRunner pid=2026458) 'kl_penalty': 'kl', (TaskRunner pid=2026458) 'lam': 1.0, (TaskRunner pid=2026458) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2026458) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2026458) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2026458) 'loss_type': 'ppo_clip', (TaskRunner pid=2026458) 'rollout_is': 'token', (TaskRunner pid=2026458) 'rollout_is_batch_normalize': False, (TaskRunner pid=2026458) 'rollout_is_threshold': 2.0, (TaskRunner pid=2026458) 'rollout_rs': None, (TaskRunner pid=2026458) 'rollout_rs_threshold': None}, (TaskRunner pid=2026458) 'use_kl_in_reward': False, (TaskRunner pid=2026458) 'use_pf_ppo': False}, (TaskRunner pid=2026458) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2026458) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2026458) 'cliprange_value': 0.5, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'enable': None, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'forward_micro_batch_size': None, (TaskRunner pid=2026458) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-05, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': -1, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False}, (TaskRunner pid=2026458) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2026458) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2026458) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2026458) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'dataloader_num_workers': 8, (TaskRunner pid=2026458) 'filter_overlong_prompts': True, (TaskRunner pid=2026458) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2026458) 'image_key': 'images', (TaskRunner pid=2026458) 'image_patch_size': 14, (TaskRunner pid=2026458) 'max_prompt_length': 2048, (TaskRunner pid=2026458) 'max_response_length': 8192, (TaskRunner pid=2026458) 'prompt_key': 'prompt', (TaskRunner pid=2026458) 'return_full_prompt': False, (TaskRunner pid=2026458) 'return_multi_modal_inputs': True, (TaskRunner pid=2026458) 'return_raw_chat': True, (TaskRunner pid=2026458) 'return_raw_input_ids': False, (TaskRunner pid=2026458) 'reward_fn_key': 'data_source', (TaskRunner pid=2026458) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2026458) 'seed': None, (TaskRunner pid=2026458) 'shuffle': True, (TaskRunner pid=2026458) 'tokenizer': None, (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'train_batch_size': 32, (TaskRunner pid=2026458) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2026458) 'train_max_samples': 3200, (TaskRunner pid=2026458) 'truncation': 'error', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_shm': False, (TaskRunner pid=2026458) 'val_batch_size': None, (TaskRunner pid=2026458) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2026458) 'val_max_samples': -1, (TaskRunner pid=2026458) 'validation_shuffle': False, (TaskRunner pid=2026458) 'video_key': 'videos'}, (TaskRunner pid=2026458) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2026458) 'capture-range-end': None, (TaskRunner pid=2026458) 'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'kill': 'none', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2026458) 'torch_memory': {'context': 'all', (TaskRunner pid=2026458) 'kw_args': {}, (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'stacks': 'all', (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2026458) 'profile_continuous_steps': False, (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'steps': None, (TaskRunner pid=2026458) 'tool': None}, (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', (TaskRunner pid=2026458) 'include_dashboard': False, (TaskRunner pid=2026458) 'num_cpus': None, (TaskRunner pid=2026458) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'USER': 'root'}}}, (TaskRunner pid=2026458) 'timeline_json_file': None}, (TaskRunner pid=2026458) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2026458) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2026458) 'name': 'custom_reward_manager', (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'name': 'naive', (TaskRunner pid=2026458) 'source': 'register'}, (TaskRunner pid=2026458) 'reward_model': {'enable': False, (TaskRunner pid=2026458) 'enable_resource_pool': False, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'launch_reward_fn_async': False, (TaskRunner pid=2026458) 'max_length': None, (TaskRunner pid=2026458) 'micro_batch_size': None, (TaskRunner pid=2026458) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'model': {'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2026458) 'trust_remote_code': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 0, (TaskRunner pid=2026458) 'num_workers': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'reward_loop_class_name': None, (TaskRunner pid=2026458) 'reward_loop_module_path': None, (TaskRunner pid=2026458) 'reward_loop_source': 'register', (TaskRunner pid=2026458) 'reward_manager': 'naive', (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enforce_eager': True, (TaskRunner pid=2026458) 'engine_kwargs': {}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2026458) 'limit_images': None, (TaskRunner pid=2026458) 'load_format': 'auto', (TaskRunner pid=2026458) 'max_model_len': None, (TaskRunner pid=2026458) 'max_num_batched_tokens': 8192, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'name': '???', (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'response_length': 2048, (TaskRunner pid=2026458) 'skip_tokenizer_init': False, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2026458) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2026458) 'memory_limit_mb': 1024, (TaskRunner pid=2026458) 'url': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_reward_loop': False}, (TaskRunner pid=2026458) 'trainer': {'balance_batch': True, (TaskRunner pid=2026458) 'critic_warmup': 0, (TaskRunner pid=2026458) 'default_hdfs_dir': None, (TaskRunner pid=2026458) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'del_local_ckpt_after_load': False, (TaskRunner pid=2026458) 'device': 'cuda', (TaskRunner pid=2026458) 'esi_redundant_time': 0, (TaskRunner pid=2026458) 'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2026458) 'log_val_generations': 0, (TaskRunner pid=2026458) 'logger': ['console', 'wandb'], (TaskRunner pid=2026458) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2026458) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 1, (TaskRunner pid=2026458) 'project_name': 'SDPO-root', (TaskRunner pid=2026458) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2026458) 'resume_from_path': None, (TaskRunner pid=2026458) 'resume_mode': 'auto', (TaskRunner pid=2026458) 'rollout_data_dir': None, (TaskRunner pid=2026458) 'save_freq': 10, (TaskRunner pid=2026458) 'test_freq': 10, (TaskRunner pid=2026458) 'total_epochs': 30, (TaskRunner pid=2026458) 'total_training_steps': 100, (TaskRunner pid=2026458) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2026458) 'val_before_train': False, (TaskRunner pid=2026458) 'val_only': False, (TaskRunner pid=2026458) 'validation_data_dir': None}, (TaskRunner pid=2026458) 'transfer_queue': {'enable': False}, (TaskRunner pid=2026458) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'dir': '/users/root/SDPO', (TaskRunner pid=2026458) 'log_dir': '/users/root/output', (TaskRunner pid=2026458) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2026458) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) use_critic=need_critic(config), (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2026458) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,289:'(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,290:Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) Using dataset class: RLHFDataset (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2026458) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2026458) dataset len: 1890 (TaskRunner pid=2026458) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:12,124:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2026458) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 301.55 examples/s] (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) self.use_critic = need_critic(self.config) (WorkerDict pid=2026872) [W702 06:19:21.430329774 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2026871) [Gloo] Rank 3 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2026873) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026873) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026869) [W702 06:19:22.501122826 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026873) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2026871) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026870) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] [repeated 12x across cluster] (WorkerDict pid=2026870) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2026868) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 3.00s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 2x across cluster] (WorkerDict pid=2026868) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2026868) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026868) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2026868) Actor use_remove_padding=True (WorkerDict pid=2026868) Actor use_fused_kernels=False (WorkerDict pid=2026868) Actor use_prefix_grouper=False (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 1 peer ranks. Expected number of connected peer ranks is : 1 (WorkerDict pid=2026871) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2026871) warnings.warn( (WorkerDict pid=2026869) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2026869) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2026868) reference model: Qwen/Qwen3-4B (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026871) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) Ref use_remove_padding=True (WorkerDict pid=2026868) Ref use_fused_kernels=False (WorkerDict pid=2026868) Ref use_prefix_grouper=False (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026874) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] [repeated 12x across cluster] (WorkerDict pid=2026873) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.89s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2026874) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 4x across cluster] (vLLMHttpServer pid=2027750) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2027750) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2027750) ['serve', (vLLMHttpServer pid=2027750) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2027750) '--dtype', (vLLMHttpServer pid=2027750) 'bfloat16', (vLLMHttpServer pid=2027750) '--load_format', (vLLMHttpServer pid=2027750) 'dummy', (vLLMHttpServer pid=2027750) '--trust_remote_code', (vLLMHttpServer pid=2027750) '--max_model_len', (vLLMHttpServer pid=2027750) '40960', (vLLMHttpServer pid=2027750) '--max_num_seqs', (vLLMHttpServer pid=2027750) '1024', (vLLMHttpServer pid=2027750) '--enable_chunked_prefill', (vLLMHttpServer pid=2027750) '--max_num_batched_tokens', (vLLMHttpServer pid=2027750) '10240', (vLLMHttpServer pid=2027750) '--enable_prefix_caching', (vLLMHttpServer pid=2027750) '--enable_sleep_mode', (vLLMHttpServer pid=2027750) '--logprobs_mode', (vLLMHttpServer pid=2027750) 'processed_logprobs', (vLLMHttpServer pid=2027750) '--disable_custom_all_reduce', (vLLMHttpServer pid=2027750) '--gpu_memory_utilization', (vLLMHttpServer pid=2027750) '0.8', (vLLMHttpServer pid=2027750) '--disable_log_stats', (vLLMHttpServer pid=2027750) '--tensor_parallel_size', (vLLMHttpServer pid=2027750) '2', (vLLMHttpServer pid=2027750) '--seed', (vLLMHttpServer pid=2027750) '0', (vLLMHttpServer pid=2027750) '--override_generation_config', (vLLMHttpServer pid=2027750) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2027750) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2027750) '--hf_overrides', (vLLMHttpServer pid=2027750) '{}', (vLLMHttpServer pid=2027750) '--scheduling_policy', (vLLMHttpServer pid=2027750) 'fcfs'] (WorkerDict pid=2026872) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026872) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (vLLMHttpServer pid=2027750) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2027750) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2027750) WARNING 07-02 06:20:18 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2026868) WARNING 07-02 06:20:26 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2027751) WARNING 07-02 06:20:19 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2026872) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026868) 2026-07-02 06:20:40,724 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2026868) 2026-07-02 06:20:40,743 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2027753) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2027753) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2027753) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2027753) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a benzene ring substituted with a chlorine atom and a trifluoromethyl group (C(F)(F)F). The key part of the product is the "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests that the reactant must contain a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a ketone group (O=C) and a nitrogen-containing ring, but it does not match the specific substituents in the product. (TaskRunner pid=2026458) - Option B contains a chlorine atom and a nitrogen, but it does not include the trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) - Option C contains a trifluoromethyl group (C(F)(F)F) and a chlorine atom, and it includes a structure that matches the "Cc2ccc(Cl)cc2" part of the product. (TaskRunner pid=2026458) - Option D contains a trifluoromethyl group but lacks the chlorine atom in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C is the only one that includes both the chlorine and the trifluoromethyl group in the correct positions, matching the structure of the product. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2026458) step:10 - global_seqlen/min:17490 - global_seqlen/max:20901 - global_seqlen/minmax_diff:3411 - global_seqlen/balanced_min:19357 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19364.75 - actor/entropy:0.24537719786167145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744035601615906 - training/rollout_probs_diff_mean:0.005785205401480198 - training/rollout_probs_diff_std:0.014700246043503284 - training/rollout_actor_probs_pearson_corr:0.9979208707809448 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9998391270637512 - rollout_corr/rollout_is_ratio_fraction_high:1.1633588655968197e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.980152829783037e-05 - rollout_corr/rollout_is_max:2.010491132736206 - rollout_corr/rollout_is_min:0.23784410953521729 - rollout_corr/rollout_is_std:0.04395073652267456 - rollout_corr/rollout_is_eff_sample_size:0.9980714630669532 - rollout_corr/rollout_is_seq_mean:0.9998910427093506 - rollout_corr/rollout_is_seq_std:0.0027966259513050318 - rollout_corr/rollout_is_seq_max:1.0104008913040161 - rollout_corr/rollout_is_seq_min:0.9919784069061279 - rollout_corr/rollout_is_seq_max_deviation:0.010400891304016113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026330736465752) - rollout_corr/training_log_ppl:np.float64(0.26078601760673337) - rollout_corr/kl:np.float64(0.0013049291076470126) - rollout_corr/k3_kl:np.float64(0.0011964667813799679) - rollout_corr/rollout_ppl:np.float64(1.3009595507755876) - rollout_corr/rollout_log_ppl:np.float64(0.25948108817101456) - rollout_corr/log_ppl_diff:np.float64(0.0013049294357188046) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024894755915738642) - rollout_corr/log_ppl_diff_max:np.float64(0.017253637313842773) - rollout_corr/log_ppl_diff_min:np.float64(-0.008967161178588867) - rollout_corr/ppl_ratio:np.float64(1.0013103527016938) - rollout_corr/chi2_token:np.float64(0.002327824244275689) - rollout_corr/chi2_seq:np.float64(0.852935308823362) - actor/pg_loss:np.float64(0.00642693554982543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010866172510759498) - actor/ppo_kl:np.float64(0.00015771930741692586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(177288.61759452336) - self_distillation/token_reweight_w_std:np.float64(2583470.6754472964) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025011864490807) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1041747338604182) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.5560944229364395) - perf/mfu/actor:np.float64(0.03574857500362935) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(141.2105484008789) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/score/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11915383713937476) - val-core/sciknoweval/acc/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16058361739063698) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7451095238095238) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04477536267518606) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.46405714285714283) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994571428571428) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015133869749850571) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997476190476191) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0010668260935279934) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0019790054535129544) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999190476190476) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006155763481184213) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981333333333333) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023247461032216924) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970238095238095) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023053472298853674) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001270969514735043 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001270969514735043 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.7734375 - response_length/max:1122.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.7734375 - response_length_non_aborted/max:1122.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7655776143074036e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4987823646515608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.95271722227335) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.580353087221738) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.95271722227335) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1122 - timing_s/gen:13.539362616837025 - timing_s/reward:0.05532005988061428 - timing_s/old_log_prob:2.3085584342479706 - timing_s/adv:0.48644337616860867 - timing_s/update_actor:15.44553491845727 - timing_s/step:31.924703411757946 - timing_s/testing:117.72085026092827 - timing_s/save_checkpoint:6.645242588594556 - timing_s/stop_profile:0.0001191999763250351 - timing_per_token_ms/adv:0.0031400055265921887 - timing_per_token_ms/gen:0.1575113731919894 - timing_per_token_ms/update_actor:0.09970135761149299 - perf/total_num_tokens:154918 - perf/time_per_step:31.924703411757946 - perf/throughput:606.5757213226894 (TaskRunner pid=2026458) Training Progress: 10%|█ | 10/100 [08:48<1:56:19, 77.55s/it] (TaskRunner pid=2026458) step:11 - global_seqlen/min:17717 - global_seqlen/max:22105 - global_seqlen/minmax_diff:4388 - global_seqlen/balanced_min:20139 - global_seqlen/balanced_max:20427 - global_seqlen/mean:20176.875 - actor/entropy:0.28003615140914917 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2986578345298767 - training/rollout_probs_diff_mean:0.006229304242879152 - training/rollout_probs_diff_std:0.014922214671969414 - training/rollout_actor_probs_pearson_corr:0.9980378746986389 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9996895790100098 - rollout_corr/rollout_is_ratio_fraction_high:2.3571841666125692e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.071552681736648e-05 - rollout_corr/rollout_is_max:2.091331720352173 - rollout_corr/rollout_is_min:0.056948471814394 - rollout_corr/rollout_is_std:0.04615355655550957 - rollout_corr/rollout_is_eff_sample_size:0.9978729526678736 - rollout_corr/rollout_is_seq_mean:0.9996609091758728 - rollout_corr/rollout_is_seq_std:0.002769676735624671 - rollout_corr/rollout_is_seq_max:1.0097713470458984 - rollout_corr/rollout_is_seq_min:0.9914740920066833 - rollout_corr/rollout_is_seq_max_deviation:0.009771347045898438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3445568550378084) - rollout_corr/training_log_ppl:np.float64(0.2928216397704091) - rollout_corr/kl:np.float64(0.0013540790551616055) - rollout_corr/k3_kl:np.float64(0.001210083299042708) - rollout_corr/rollout_ppl:np.float64(1.3426842936314642) - rollout_corr/rollout_log_ppl:np.float64(0.2914675589709077) - rollout_corr/log_ppl_diff:np.float64(0.0013540807995013893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025300010456703603) - rollout_corr/log_ppl_diff_max:np.float64(0.011778771877288818) - rollout_corr/log_ppl_diff_min:np.float64(-0.006693422794342041) - rollout_corr/ppl_ratio:np.float64(1.001359449699521) - rollout_corr/chi2_token:np.float64(0.0021713071037083864) - rollout_corr/chi2_seq:np.float64(0.5860715808812529) - actor/pg_loss:np.float64(0.009074105182662606) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010309692956980143) - actor/ppo_kl:np.float64(-0.00014117327255291912) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(239610.312335568) - self_distillation/token_reweight_w_std:np.float64(3316290.997419097) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019973907619715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14387543653720059) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.6877006888389587) - perf/mfu/actor:np.float64(0.037142691887104826) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9859390258789) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006922754924744368 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006922754924744368 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.43359375 - response_length/max:1206.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.43359375 - response_length_non_aborted/max:1206.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012538395822048187 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7413379289209843) - timing_s/agent_loop/generate_sequences/max:np.float64(8.170563256368041) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.67358700458135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.170563256368041) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1206 - timing_s/gen:14.099589532241225 - timing_s/reward:0.05785256810486317 - timing_s/old_log_prob:2.514467492699623 - timing_s/adv:0.5351101234555244 - timing_s/update_actor:15.359056254848838 - timing_s/step:32.6669915933162 - timing_s/stop_profile:0.0001175031065940857 - timing_per_token_ms/adv:0.003315120177527023 - timing_per_token_ms/gen:0.16617664186407563 - timing_per_token_ms/update_actor:0.09515259582349124 - perf/total_num_tokens:161415 - perf/time_per_step:32.6669915933162 - perf/throughput:617.6532951423746 (TaskRunner pid=2026458) Training Progress: 11%|█ | 11/100 [09:20<1:34:40, 63.83s/it] (TaskRunner pid=2026458) step:12 - global_seqlen/min:17937 - global_seqlen/max:22358 - global_seqlen/minmax_diff:4421 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:19763 - global_seqlen/mean:19757.25 - actor/entropy:0.2599850594997406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29290565848350525 - training/rollout_probs_diff_mean:0.00601433590054512 - training/rollout_probs_diff_std:0.014930167235434055 - training/rollout_actor_probs_pearson_corr:0.9979494214057922 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999984085559845 - rollout_corr/rollout_is_ratio_fraction_high:2.223754199803807e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.33563111780677e-05 - rollout_corr/rollout_is_max:2.084258556365967 - rollout_corr/rollout_is_min:0.48210135102272034 - rollout_corr/rollout_is_std:0.04539177566766739 - rollout_corr/rollout_is_eff_sample_size:0.9979434670549656 - rollout_corr/rollout_is_seq_mean:0.9999362230300903 - rollout_corr/rollout_is_seq_std:0.002637566300109029 - rollout_corr/rollout_is_seq_max:1.0081006288528442 - rollout_corr/rollout_is_seq_min:0.9917522072792053 - rollout_corr/rollout_is_seq_max_deviation:0.008247792720794678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3254084084182978) - rollout_corr/training_log_ppl:np.float64(0.2779389991774224) - rollout_corr/kl:np.float64(0.0013704395118168122) - rollout_corr/k3_kl:np.float64(0.0011261843454803966) - rollout_corr/rollout_ppl:np.float64(1.3235342632979155) - rollout_corr/rollout_log_ppl:np.float64(0.27656855969689786) - rollout_corr/log_ppl_diff:np.float64(0.0013704394805245101) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024474224192090333) - rollout_corr/log_ppl_diff_max:np.float64(0.011240929365158081) - rollout_corr/log_ppl_diff_min:np.float64(-0.010027527809143066) - rollout_corr/ppl_ratio:np.float64(1.001375619089231) - rollout_corr/chi2_token:np.float64(0.001807245658710599) - rollout_corr/chi2_seq:np.float64(1.2864351340103894) - actor/pg_loss:np.float64(0.0068493077997118235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010652919690983254) - actor/ppo_kl:np.float64(0.00020145679512317827) - actor/pg_clipfrac_lower:np.float64(1.669337689236272e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(145999.17218339257) - self_distillation/token_reweight_w_std:np.float64(2374372.460695313) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004515508189797) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1246801737579517) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5914773046970367) - perf/mfu/actor:np.float64(0.03625096042724267) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.91592407226562) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004239031113684177 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004239031113684177 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:351.3203125 - response_length/max:1148.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:351.3203125 - response_length_non_aborted/max:1148.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.013755116611719131 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6791318338364363) - timing_s/agent_loop/generate_sequences/max:np.float64(8.12350033968687) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.846359725270304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.12350033968687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1148 - timing_s/gen:13.899831658229232 - timing_s/reward:0.05922466889023781 - timing_s/old_log_prob:2.4094871655106544 - timing_s/adv:0.5697984360158443 - timing_s/update_actor:15.474863145500422 - timing_s/step:32.48048596084118 - timing_s/stop_profile:0.0001192968338727951 - timing_per_token_ms/adv:0.003604995862378648 - timing_per_token_ms/gen:0.15454904109752532 - timing_per_token_ms/update_actor:0.09790623154475206 - perf/total_num_tokens:158058 - perf/time_per_step:32.48048596084118 - perf/throughput:608.2806157463147 (TaskRunner pid=2026458) Training Progress: 12%|█▏ | 12/100 [09:53<1:19:39, 54.31s/it] (TaskRunner pid=2026458) step:13 - global_seqlen/min:17460 - global_seqlen/max:22387 - global_seqlen/minmax_diff:4927 - global_seqlen/balanced_min:19538 - global_seqlen/balanced_max:19656 - global_seqlen/mean:19562.0 - actor/entropy:0.28330478072166443 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932348847389221 - training/rollout_probs_diff_mean:0.006090556271374226 - training/rollout_probs_diff_std:0.014429626986384392 - training/rollout_actor_probs_pearson_corr:0.998181939125061 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9998684525489807 - rollout_corr/rollout_is_ratio_fraction_high:2.3373224394163117e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.3373224394163117e-05 - rollout_corr/rollout_is_max:2.1884407997131348 - rollout_corr/rollout_is_min:0.4281519055366516 - rollout_corr/rollout_is_std:0.04431807994842529 - rollout_corr/rollout_is_eff_sample_size:0.9980391641302073 - rollout_corr/rollout_is_seq_mean:0.9998126029968262 - rollout_corr/rollout_is_seq_std:0.0026776245795190334 - rollout_corr/rollout_is_seq_max:1.009365439414978 - rollout_corr/rollout_is_seq_min:0.9913782477378845 - rollout_corr/rollout_is_seq_max_deviation:0.009365439414978027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3591533433645964) - rollout_corr/training_log_ppl:np.float64(0.30308022926328704) - rollout_corr/kl:np.float64(0.0012214116412607723) - rollout_corr/k3_kl:np.float64(0.0011643693818541578) - rollout_corr/rollout_ppl:np.float64(1.3574783634394407) - rollout_corr/rollout_log_ppl:np.float64(0.3018588193517644) - rollout_corr/log_ppl_diff:np.float64(0.0012214099115226418) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024031201901379973) - rollout_corr/log_ppl_diff_max:np.float64(0.011142253875732422) - rollout_corr/log_ppl_diff_min:np.float64(-0.00809323787689209) - rollout_corr/ppl_ratio:np.float64(1.0012265297118574) - rollout_corr/chi2_token:np.float64(0.0022032533306628466) - rollout_corr/chi2_seq:np.float64(1.1880777331534773) - actor/pg_loss:np.float64(0.008368437876924872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011567205356186605) - actor/ppo_kl:np.float64(-1.6140308314760432e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(93498.56042674859) - self_distillation/token_reweight_w_std:np.float64(1510537.690864286) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999996354104951) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12338341507711448) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.620448037981987) - perf/mfu/actor:np.float64(0.036300476348807696) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.69003295898438) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003617006354033947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003617006354033947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.25 - response_length/max:1159.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.25 - response_length_non_aborted/max:1159.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014793872833251953 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3433514721691608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.973012773320079) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6053173123218585) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.973012773320079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:1159 - timing_s/gen:13.718764901161194 - timing_s/reward:0.05917247012257576 - timing_s/old_log_prob:2.3764582853764296 - timing_s/adv:0.49192045256495476 - timing_s/update_actor:15.186074122786522 - timing_s/step:31.921324100345373 - timing_s/stop_profile:0.00019265897572040558 - timing_per_token_ms/adv:0.003143342018741404 - timing_per_token_ms/gen:0.1603258800154403 - timing_per_token_ms/update_actor:0.09703809760496448 - perf/total_num_tokens:156496 - perf/time_per_step:31.921324100345373 - perf/throughput:612.8191906609648 (TaskRunner pid=2026458) Training Progress: 13%|█▎ | 13/100 [10:25<1:08:56, 47.54s/it] (TaskRunner pid=2026458) step:14 - global_seqlen/min:16941 - global_seqlen/max:22560 - global_seqlen/minmax_diff:5619 - global_seqlen/balanced_min:18841 - global_seqlen/balanced_max:18844 - global_seqlen/mean:18842.75 - actor/entropy:0.2922348976135254 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466010570526123 - training/rollout_probs_diff_mean:0.006264000199735165 - training/rollout_probs_diff_std:0.014866359531879425 - training/rollout_actor_probs_pearson_corr:0.9981194138526917 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002233982086182 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.292350346688181e-05 - rollout_corr/rollout_is_max:1.9629987478256226 - rollout_corr/rollout_is_min:0.12186871469020844 - rollout_corr/rollout_is_std:0.04516667500138283 - rollout_corr/rollout_is_eff_sample_size:0.9979653119171001 - rollout_corr/rollout_is_seq_mean:1.0002156496047974 - rollout_corr/rollout_is_seq_std:0.00266577722504735 - rollout_corr/rollout_is_seq_max:1.0086764097213745 - rollout_corr/rollout_is_seq_min:0.9924325942993164 - rollout_corr/rollout_is_seq_max_deviation:0.008676409721374512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.356894207186997) - rollout_corr/training_log_ppl:np.float64(0.30215946782846004) - rollout_corr/kl:np.float64(0.0010239662146345752) - rollout_corr/k3_kl:np.float64(0.0012049657070747344) - rollout_corr/rollout_ppl:np.float64(1.3555219103582203) - rollout_corr/rollout_log_ppl:np.float64(0.3011355001362972) - rollout_corr/log_ppl_diff:np.float64(0.0010239676921628416) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023661727900616825) - rollout_corr/log_ppl_diff_max:np.float64(0.017396926879882812) - rollout_corr/log_ppl_diff_min:np.float64(-0.007369279861450195) - rollout_corr/ppl_ratio:np.float64(1.0010292823426425) - rollout_corr/chi2_token:np.float64(0.0027081489097326994) - rollout_corr/chi2_seq:np.float64(1.2070772459264845) - actor/pg_loss:np.float64(0.00797818333376199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010150298585358541) - actor/ppo_kl:np.float64(0.00015036190526984683) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.453125) - self_distillation/token_reweight_w_mean:np.float64(119982.02733938815) - self_distillation/token_reweight_w_std:np.float64(1933813.0637409594) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014771707355976) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1388165877724532) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.6455883979797363) - perf/mfu/actor:np.float64(0.034641753809947065) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.4944953918457) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480345986783504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480345986783504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.3984375 - response_length/max:682.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.3984375 - response_length_non_aborted/max:682.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012210384011268616 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7070676069706678) - timing_s/agent_loop/generate_sequences/max:np.float64(5.696014670655131) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4708188238437288) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.696014670655131) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:413 - timing_s/agent_loop/slowest/response_length:682 - timing_s/gen:11.448512280359864 - timing_s/reward:0.05713308043777943 - timing_s/old_log_prob:2.4212917368859053 - timing_s/adv:0.4922778233885765 - timing_s/update_actor:15.44989874586463 - timing_s/step:29.956744581460953 - timing_s/stop_profile:0.00011506862938404083 - timing_per_token_ms/adv:0.003265697837288722 - timing_per_token_ms/gen:0.1391442704047238 - timing_per_token_ms/update_actor:0.10249232958209809 - perf/total_num_tokens:150742 - perf/time_per_step:29.956744581460953 - perf/throughput:628.9985865707529 (TaskRunner pid=2026458) Training Progress: 14%|█▍ | 14/100 [10:55<1:00:33, 42.25s/it] (TaskRunner pid=2026458) step:15 - global_seqlen/min:17798 - global_seqlen/max:24465 - global_seqlen/minmax_diff:6667 - global_seqlen/balanced_min:20570 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20572.75 - actor/entropy:0.28748324513435364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564783573150635 - training/rollout_probs_diff_mean:0.005869686137884855 - training/rollout_probs_diff_std:0.014366100542247295 - training/rollout_actor_probs_pearson_corr:0.9982949495315552 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001139640808105 - rollout_corr/rollout_is_ratio_fraction_high:5.4864267440279946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3891413952223957e-05 - rollout_corr/rollout_is_max:3.0497446060180664 - rollout_corr/rollout_is_min:0.38640856742858887 - rollout_corr/rollout_is_std:0.045057009905576706 - rollout_corr/rollout_is_eff_sample_size:0.9979744538008674 - rollout_corr/rollout_is_seq_mean:1.0000189542770386 - rollout_corr/rollout_is_seq_std:0.002783029805868864 - rollout_corr/rollout_is_seq_max:1.0093011856079102 - rollout_corr/rollout_is_seq_min:0.9919822812080383 - rollout_corr/rollout_is_seq_max_deviation:0.009301185607910156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3641661452129483) - rollout_corr/training_log_ppl:np.float64(0.3061258476227522) - rollout_corr/kl:np.float64(0.0008984394618103408) - rollout_corr/k3_kl:np.float64(0.0010934443635051139) - rollout_corr/rollout_ppl:np.float64(1.3629517373628914) - rollout_corr/rollout_log_ppl:np.float64(0.3052274066139944) - rollout_corr/log_ppl_diff:np.float64(0.0008984410087577999) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023149027838371694) - rollout_corr/log_ppl_diff_max:np.float64(0.015007197856903076) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058860182762146) - rollout_corr/ppl_ratio:np.float64(1.00090327905491) - rollout_corr/chi2_token:np.float64(0.0026275489944964647) - rollout_corr/chi2_seq:np.float64(1.554575996240601) - actor/pg_loss:np.float64(0.006172901834361255) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010921142766164849) - actor/ppo_kl:np.float64(-0.00014767027690254508) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(168825.68510642461) - self_distillation/token_reweight_w_std:np.float64(2411026.6848952193) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999142882879823) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13169058857602067) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5341654643416405) - perf/mfu/actor:np.float64(0.03786336781112096) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.43387985229492) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036114403046667576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036114403046667576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.9921875 - response_length/max:841.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.9921875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000149507075548172 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8174101784825325) - timing_s/agent_loop/generate_sequences/max:np.float64(6.822324391454458) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8521247571407002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.822324391454458) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:187 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.53348901681602 - timing_s/reward:0.06065330095589161 - timing_s/old_log_prob:2.2750260774046183 - timing_s/adv:0.5170386601239443 - timing_s/update_actor:15.529155055060983 - timing_s/step:31.002764070406556 - timing_s/stop_profile:0.00011553429067134857 - timing_per_token_ms/adv:0.003141526170079014 - timing_per_token_ms/gen:0.13752813458002525 - timing_per_token_ms/update_actor:0.09435512422416172 - perf/total_num_tokens:164582 - perf/time_per_step:31.002764070406556 - perf/throughput:663.5779298026383 (TaskRunner pid=2026458) Training Progress: 15%|█▌ | 15/100 [11:26<55:04, 38.87s/it] (TaskRunner pid=2026458) step:16 - global_seqlen/min:17436 - global_seqlen/max:21462 - global_seqlen/minmax_diff:4026 - global_seqlen/balanced_min:19472 - global_seqlen/balanced_max:19474 - global_seqlen/mean:19473.375 - actor/entropy:0.29451751708984375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34674251079559326 - training/rollout_probs_diff_mean:0.006398364901542664 - training/rollout_probs_diff_std:0.014944251626729965 - training/rollout_actor_probs_pearson_corr:0.9980317950248718 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.64453125 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.64453125 - rollout_corr/rollout_is_mean:1.000139832496643 - rollout_corr/rollout_is_ratio_fraction_high:3.57470526068937e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001429882104275748 - rollout_corr/rollout_is_max:2.075308322906494 - rollout_corr/rollout_is_min:0.33335795998573303 - rollout_corr/rollout_is_std:0.04721960052847862 - rollout_corr/rollout_is_eff_sample_size:0.9977759819851924 - rollout_corr/rollout_is_seq_mean:1.0001648664474487 - rollout_corr/rollout_is_seq_std:0.002871463308110833 - rollout_corr/rollout_is_seq_max:1.0102380514144897 - rollout_corr/rollout_is_seq_min:0.9917946457862854 - rollout_corr/rollout_is_seq_max_deviation:0.010238051414489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36143590323627) - rollout_corr/training_log_ppl:np.float64(0.3041792264266405) - rollout_corr/kl:np.float64(0.0010337699030245062) - rollout_corr/k3_kl:np.float64(0.0012385786424147227) - rollout_corr/rollout_ppl:np.float64(1.3600022280588746) - rollout_corr/rollout_log_ppl:np.float64(0.30314545703004114) - rollout_corr/log_ppl_diff:np.float64(0.0010337693965993822) - rollout_corr/log_ppl_abs_diff:np.float64(0.002399776072707027) - rollout_corr/log_ppl_diff_max:np.float64(0.011399328708648682) - rollout_corr/log_ppl_diff_min:np.float64(-0.006810277700424194) - rollout_corr/ppl_ratio:np.float64(1.0010383201297373) - rollout_corr/chi2_token:np.float64(0.0029240259900689125) - rollout_corr/chi2_seq:np.float64(1.0353804978076369) - actor/pg_loss:np.float64(0.007192879682406783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010274524142914743) - actor/ppo_kl:np.float64(4.0873823671461196e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.64453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(90371.76209130394) - self_distillation/token_reweight_w_std:np.float64(1389709.4737052019) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000855000689626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11968303663888946) - self_distillation/empty_target_batch:np.float64(0.35546875) - actor/grad_norm:np.float64(0.6043514311313629) - perf/mfu/actor:np.float64(0.03566211221390109) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.67253494262695) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006435959134250879 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006435959134250879 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.82421875 - response_length/max:698.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.82421875 - response_length_non_aborted/max:698.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010755844414234161 - timing_s/agent_loop/generate_sequences/min:np.float64(1.786530127748847) - timing_s/agent_loop/generate_sequences/max:np.float64(5.909618901088834) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.657506644296518) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.909618901088834) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:670 - timing_s/gen:11.721529113128781 - timing_s/reward:0.05861745588481426 - timing_s/old_log_prob:2.29558939486742 - timing_s/adv:0.5112862680107355 - timing_s/update_actor:15.580748109146953 - timing_s/step:30.256891636177897 - timing_s/stop_profile:0.00016321614384651184 - timing_per_token_ms/adv:0.003281957210875975 - timing_per_token_ms/gen:0.13967004412531464 - timing_per_token_ms/update_actor:0.10001314685530213 - perf/total_num_tokens:155787 - perf/time_per_step:30.256891636177897 - perf/throughput:643.6013069074107 (TaskRunner pid=2026458) Training Progress: 16%|█▌ | 16/100 [11:56<50:48, 36.29s/it] (TaskRunner pid=2026458) step:17 - global_seqlen/min:19903 - global_seqlen/max:25273 - global_seqlen/minmax_diff:5370 - global_seqlen/balanced_min:22184 - global_seqlen/balanced_max:22190 - global_seqlen/mean:22187.375 - actor/entropy:0.28030383586883545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4096241295337677 - training/rollout_probs_diff_mean:0.00557515537366271 - training/rollout_probs_diff_std:0.014061576686799526 - training/rollout_actor_probs_pearson_corr:0.9982938766479492 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.640625 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.640625 - rollout_corr/rollout_is_mean:1.000001311302185 - rollout_corr/rollout_is_ratio_fraction_high:2.1693151211366057e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4232874390436336e-05 - rollout_corr/rollout_is_max:3.007925271987915 - rollout_corr/rollout_is_min:0.3316154181957245 - rollout_corr/rollout_is_std:0.0431625172495842 - rollout_corr/rollout_is_eff_sample_size:0.9981403427818611 - rollout_corr/rollout_is_seq_mean:1.0001276731491089 - rollout_corr/rollout_is_seq_std:0.002614374505355954 - rollout_corr/rollout_is_seq_max:1.0077310800552368 - rollout_corr/rollout_is_seq_min:0.9923279881477356 - rollout_corr/rollout_is_seq_max_deviation:0.007731080055236816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.381326341535896) - rollout_corr/training_log_ppl:np.float64(0.3175459819030948) - rollout_corr/kl:np.float64(0.0007249800520758853) - rollout_corr/k3_kl:np.float64(0.0011385135364321286) - rollout_corr/rollout_ppl:np.float64(1.3803315898403525) - rollout_corr/rollout_log_ppl:np.float64(0.316821001295466) - rollout_corr/log_ppl_diff:np.float64(0.0007249806076288223) - rollout_corr/log_ppl_abs_diff:np.float64(0.00215923844370991) - rollout_corr/log_ppl_diff_max:np.float64(0.00994020700454712) - rollout_corr/log_ppl_diff_min:np.float64(-0.007171809673309326) - rollout_corr/ppl_ratio:np.float64(1.0007290567737073) - rollout_corr/chi2_token:np.float64(0.003144999034702778) - rollout_corr/chi2_seq:np.float64(0.9032414378598332) - actor/pg_loss:np.float64(0.0055801779963076115) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008305092796945246) - actor/ppo_kl:np.float64(-0.00022399483597723702) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(213885.15984977316) - self_distillation/token_reweight_w_std:np.float64(3187382.02780142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009852752555162) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11256717666401528) - self_distillation/empty_target_batch:np.float64(0.359375) - actor/grad_norm:np.float64(0.5271423608064651) - perf/mfu/actor:np.float64(0.04032452755142399) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.51458740234375) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036620749160647392 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036620749160647392 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.13671875 - response_length/max:1140.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.13671875 - response_length_non_aborted/max:1140.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014656782150268555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6168197989463806) - timing_s/agent_loop/generate_sequences/max:np.float64(8.074994288384914) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7389801555254962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.074994288384914) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1140 - timing_s/gen:13.71630746126175 - timing_s/reward:0.06177904084324837 - timing_s/old_log_prob:2.311936927959323 - timing_s/adv:0.5291431322693825 - timing_s/update_actor:15.853100046515465 - timing_s/step:32.560671512037516 - timing_s/stop_profile:0.00012019649147987366 - timing_per_token_ms/adv:0.0029811048640802622 - timing_per_token_ms/gen:0.14877496026098758 - timing_per_token_ms/update_actor:0.08931374287469487 - perf/total_num_tokens:177499 - perf/time_per_step:32.560671512037516 - perf/throughput:681.4163826995226 (TaskRunner pid=2026458) Training Progress: 17%|█▋ | 17/100 [12:29<48:40, 35.18s/it] (TaskRunner pid=2026458) step:18 - global_seqlen/min:17147 - global_seqlen/max:20375 - global_seqlen/minmax_diff:3228 - global_seqlen/balanced_min:18455 - global_seqlen/balanced_max:18458 - global_seqlen/mean:18457.0 - actor/entropy:0.3130648136138916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772509455680847 - training/rollout_probs_diff_mean:0.0063330200500786304 - training/rollout_probs_diff_std:0.014608096331357956 - training/rollout_actor_probs_pearson_corr:0.9982773065567017 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9997703433036804 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.43533785478212e-05 - rollout_corr/rollout_is_max:1.9900730848312378 - rollout_corr/rollout_is_min:0.4134674072265625 - rollout_corr/rollout_is_std:0.04487474635243416 - rollout_corr/rollout_is_eff_sample_size:0.9979895323387393 - rollout_corr/rollout_is_seq_mean:0.9998025298118591 - rollout_corr/rollout_is_seq_std:0.002652442781254649 - rollout_corr/rollout_is_seq_max:1.0070677995681763 - rollout_corr/rollout_is_seq_min:0.9913365244865417 - rollout_corr/rollout_is_seq_max_deviation:0.008663475513458252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3969450276345015) - rollout_corr/training_log_ppl:np.float64(0.3299181046313606) - rollout_corr/kl:np.float64(0.001080212333377517) - rollout_corr/k3_kl:np.float64(0.0011240256234259505) - rollout_corr/rollout_ppl:np.float64(1.3953958731144667) - rollout_corr/rollout_log_ppl:np.float64(0.32883789180777967) - rollout_corr/log_ppl_diff:np.float64(0.0010802128235809505) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023538178065791726) - rollout_corr/log_ppl_diff_max:np.float64(0.011510014533996582) - rollout_corr/log_ppl_diff_min:np.float64(-0.009586125612258911) - rollout_corr/ppl_ratio:np.float64(1.0010849123355001) - rollout_corr/chi2_token:np.float64(0.0023459065705537796) - rollout_corr/chi2_seq:np.float64(0.5731152268126607) - actor/pg_loss:np.float64(0.007650406565517187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011541137814674585) - actor/ppo_kl:np.float64(-0.00018551408525979696) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(258180.61296664365) - self_distillation/token_reweight_w_std:np.float64(3445719.439609512) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022555205505341) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15918678505113348) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.6539853662252426) - perf/mfu/actor:np.float64(0.033737344150768206) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.57009506225586) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005579437594860792 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005579437594860792 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:303.5 - response_length/max:713.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.5 - response_length_non_aborted/max:713.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013168901205062866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.151022594422102) - timing_s/agent_loop/generate_sequences/max:np.float64(5.789204815402627) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2287484776461497) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.789204815402627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:713 - timing_s/gen:11.655897682532668 - timing_s/reward:0.05685456842184067 - timing_s/old_log_prob:2.3437488954514265 - timing_s/adv:0.4995372127741575 - timing_s/update_actor:15.640153270214796 - timing_s/step:30.285941103473306 - timing_s/stop_profile:0.00014618411660194397 - timing_per_token_ms/adv:0.0033831148939031095 - timing_per_token_ms/gen:0.15001927618580968 - timing_per_token_ms/update_actor:0.10592291048257298 - perf/total_num_tokens:147656 - perf/time_per_step:30.285941103473306 - perf/throughput:609.4246811397015 (TaskRunner pid=2026458) Training Progress: 18%|█▊ | 18/100 [12:59<46:05, 33.73s/it] (TaskRunner pid=2026458) step:19 - global_seqlen/min:16712 - global_seqlen/max:22903 - global_seqlen/minmax_diff:6191 - global_seqlen/balanced_min:19739 - global_seqlen/balanced_max:19742 - global_seqlen/mean:19740.875 - actor/entropy:0.30067509412765503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2716989815235138 - training/rollout_probs_diff_mean:0.006011276505887508 - training/rollout_probs_diff_std:0.014068786054849625 - training/rollout_actor_probs_pearson_corr:0.9983283281326294 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0000412464141846 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.48191299760947e-05 - rollout_corr/rollout_is_max:1.967125415802002 - rollout_corr/rollout_is_min:0.3074234426021576 - rollout_corr/rollout_is_std:0.04442286118865013 - rollout_corr/rollout_is_eff_sample_size:0.9980306147615733 - rollout_corr/rollout_is_seq_mean:1.0000803470611572 - rollout_corr/rollout_is_seq_std:0.0029041871894150972 - rollout_corr/rollout_is_seq_max:1.0110079050064087 - rollout_corr/rollout_is_seq_min:0.9903981685638428 - rollout_corr/rollout_is_seq_max_deviation:0.011007905006408691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.375831812620163) - rollout_corr/training_log_ppl:np.float64(0.31493956671329215) - rollout_corr/kl:np.float64(0.0009361723424987645) - rollout_corr/k3_kl:np.float64(0.00108607468507671) - rollout_corr/rollout_ppl:np.float64(1.3745251446962357) - rollout_corr/rollout_log_ppl:np.float64(0.31400339302490465) - rollout_corr/log_ppl_diff:np.float64(0.0009361736883874983) - rollout_corr/log_ppl_abs_diff:np.float64(0.002310327283339575) - rollout_corr/log_ppl_diff_max:np.float64(0.008226662874221802) - rollout_corr/log_ppl_diff_min:np.float64(-0.010857582092285156) - rollout_corr/ppl_ratio:np.float64(1.000940806698054) - rollout_corr/chi2_token:np.float64(0.002460781019181013) - rollout_corr/chi2_seq:np.float64(1.263379932148382) - actor/pg_loss:np.float64(0.005378377623856068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007215913469735824) - actor/ppo_kl:np.float64(-2.377988029422795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(92620.61167126987) - self_distillation/token_reweight_w_std:np.float64(1450653.645121337) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001238340511918) - self_distillation/token_reweight_w_clip_frac:np.float64(0.096618483774364) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.5270146429538727) - perf/mfu/actor:np.float64(0.03604524222111994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.61618423461914) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024989761877804995 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024989761877804995 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.77734375 - response_length/max:828.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.77734375 - response_length_non_aborted/max:828.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014412589371204376 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8398964125663042) - timing_s/agent_loop/generate_sequences/max:np.float64(6.303931085392833) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4352486536299693) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.303931085392833) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:828 - timing_s/gen:12.11710455827415 - timing_s/reward:0.05806134268641472 - timing_s/old_log_prob:2.3140441831201315 - timing_s/adv:0.5250083282589912 - timing_s/update_actor:15.580280201509595 - timing_s/step:30.68251054175198 - timing_s/stop_profile:0.00012233667075634003 - timing_per_token_ms/adv:0.0033243734653288625 - timing_per_token_ms/gen:0.150368000177136 - timing_per_token_ms/update_actor:0.09865494944822352 - perf/total_num_tokens:157927 - perf/time_per_step:30.68251054175198 - perf/throughput:643.3917776427427 (TaskRunner pid=2026458) Training Progress: 19%|█▉ | 19/100 [13:30<44:19, 32.83s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a structure with a substituted benzene ring (with a Cl group), a trifluoromethyl group (C(F)(F)F), and a nitrogen-containing amine group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a structure with a substituted benzene ring, a nitrogen atom, and a carbonyl group. It also includes the Cl and trifluoromethyl groups, which match the product's structure. However, it has an extra nitrogen and a different arrangement that may not be the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option B has a similar structure but with a different arrangement of substituents and a different nitrogen configuration, which does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option C includes a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a hydroxyl group and a different type of substitution, which does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option D has a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a different type of substitution and does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A is the only one that matches the structure of the product, including the correct substituents and arrangement of atoms. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2026458) step:20 - global_seqlen/min:17486 - global_seqlen/max:22655 - global_seqlen/minmax_diff:5169 - global_seqlen/balanced_min:20031 - global_seqlen/balanced_max:20228 - global_seqlen/mean:20058.625 - actor/entropy:0.297139972448349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3528301417827606 - training/rollout_probs_diff_mean:0.006166450679302216 - training/rollout_probs_diff_std:0.014436931349337101 - training/rollout_actor_probs_pearson_corr:0.9982419013977051 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000038743019104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.690288890036754e-05 - rollout_corr/rollout_is_max:1.9574382305145264 - rollout_corr/rollout_is_min:0.3535598814487457 - rollout_corr/rollout_is_std:0.04514159634709358 - rollout_corr/rollout_is_eff_sample_size:0.9979664991652689 - rollout_corr/rollout_is_seq_mean:1.0000238418579102 - rollout_corr/rollout_is_seq_std:0.0029090906027704477 - rollout_corr/rollout_is_seq_max:1.0110317468643188 - rollout_corr/rollout_is_seq_min:0.9888052940368652 - rollout_corr/rollout_is_seq_max_deviation:0.011194705963134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3683372386731207) - rollout_corr/training_log_ppl:np.float64(0.31030508255935274) - rollout_corr/kl:np.float64(0.0011023225397535086) - rollout_corr/k3_kl:np.float64(0.001434143366623175) - rollout_corr/rollout_ppl:np.float64(1.3668043981306255) - rollout_corr/rollout_log_ppl:np.float64(0.30920275836251676) - rollout_corr/log_ppl_diff:np.float64(0.0011023241968359798) - rollout_corr/log_ppl_abs_diff:np.float64(0.002514752763090655) - rollout_corr/log_ppl_diff_max:np.float64(0.0131492018699646) - rollout_corr/log_ppl_diff_min:np.float64(-0.008862733840942383) - rollout_corr/ppl_ratio:np.float64(1.0011082254350185) - rollout_corr/chi2_token:np.float64(0.0035093913320451975) - rollout_corr/chi2_seq:np.float64(1.2149556565564126) - actor/pg_loss:np.float64(0.009395288652740419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015388868646368792) - actor/ppo_kl:np.float64(4.407900779046514e-05) - actor/pg_clipfrac_lower:np.float64(3.007869418070186e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(178377.74242206803) - self_distillation/token_reweight_w_std:np.float64(2601166.713610011) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996962018776685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1573005250247661) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.68055609613657) - perf/mfu/actor:np.float64(0.03693197062950951) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.50461959838867) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/score/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/score/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/score/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11104188480484187) - val-core/sciknoweval/acc/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1512467176955187) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7826571428571428) - val-core/sciknoweval/acc/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06785839628022725) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5356190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.000300566358820181) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9992904761904762) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0016956593912030376) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9996047619047619) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0013137248630794105) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.002042357581276089) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999285714285715) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005788212276734866) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980380952380952) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023437861108329258) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9968761904761905) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002262100242179584) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011333633214235306 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011333633214235306 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.23828125 - response_length/max:1220.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.23828125 - response_length_non_aborted/max:1220.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000131312757730484 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3023900520056486) - timing_s/agent_loop/generate_sequences/max:np.float64(8.421228867024183) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7250610580740613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.421228867024183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1220 - timing_s/gen:14.119003845378757 - timing_s/reward:0.058908019214868546 - timing_s/old_log_prob:2.3049094192683697 - timing_s/adv:0.5378871113061905 - timing_s/update_actor:15.319367559626698 - timing_s/step:32.428459821268916 - timing_s/testing:112.78920979425311 - timing_s/save_checkpoint:6.872485619038343 - timing_s/stop_profile:0.00016718730330467224 - timing_per_token_ms/adv:0.003351968986571802 - timing_per_token_ms/gen:0.16068242321386106 - timing_per_token_ms/update_actor:0.09546621191399397 - perf/total_num_tokens:160469 - perf/time_per_step:32.428459821268916 - perf/throughput:618.5500363123663 (TaskRunner pid=2026458) Training Progress: 20%|██ | 20/100 [16:02<1:31:32, 68.66s/it] (TaskRunner pid=2026458) step:21 - global_seqlen/min:17387 - global_seqlen/max:23009 - global_seqlen/minmax_diff:5622 - global_seqlen/balanced_min:19296 - global_seqlen/balanced_max:19303 - global_seqlen/mean:19300.625 - actor/entropy:0.2956748902797699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42548468708992004 - training/rollout_probs_diff_mean:0.005995167884975672 - training/rollout_probs_diff_std:0.014325084164738655 - training/rollout_actor_probs_pearson_corr:0.9982481002807617 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:0.999878466129303 - rollout_corr/rollout_is_ratio_fraction_high:1.2208670341351535e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.325202022911981e-05 - rollout_corr/rollout_is_max:2.6692886352539062 - rollout_corr/rollout_is_min:0.4042225778102875 - rollout_corr/rollout_is_std:0.044611647725105286 - rollout_corr/rollout_is_eff_sample_size:0.9980132789914753 - rollout_corr/rollout_is_seq_mean:0.9997992515563965 - rollout_corr/rollout_is_seq_std:0.0029755644500255585 - rollout_corr/rollout_is_seq_max:1.0099884271621704 - rollout_corr/rollout_is_seq_min:0.9911137819290161 - rollout_corr/rollout_is_seq_max_deviation:0.00998842716217041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3673860565759242) - rollout_corr/training_log_ppl:np.float64(0.3087902419210877) - rollout_corr/kl:np.float64(0.002253606636003269) - rollout_corr/k3_kl:np.float64(0.0017760526020538236) - rollout_corr/rollout_ppl:np.float64(1.36424843640998) - rollout_corr/rollout_log_ppl:np.float64(0.30653663334669545) - rollout_corr/log_ppl_diff:np.float64(0.002253608574392274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032565151050221175) - rollout_corr/log_ppl_diff_max:np.float64(0.021895110607147217) - rollout_corr/log_ppl_diff_min:np.float64(-0.006019324064254761) - rollout_corr/ppl_ratio:np.float64(1.002263783942908) - rollout_corr/chi2_token:np.float64(0.002287261188030243) - rollout_corr/chi2_seq:np.float64(0.5934691540896893) - actor/pg_loss:np.float64(0.010032993159256876) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001869809015715873) - actor/ppo_kl:np.float64(0.000984730247550747) - actor/pg_clipfrac_lower:np.float64(2.9235126021376345e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(353075.78875684226) - self_distillation/token_reweight_w_std:np.float64(4698832.619528098) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0033803705591708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17246504218201153) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.7826535254716873) - perf/mfu/actor:np.float64(0.03582340340916603) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.28646850585938) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004340182524174452 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004340182524174452 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:319.95703125 - response_length/max:825.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.95703125 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.8999285101890564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3953026235103607) - timing_s/agent_loop/generate_sequences/max:np.float64(6.321475323289633) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4004087638677447) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.321475323289633) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:348 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.068734215572476 - timing_s/reward:0.05136189982295036 - timing_s/old_log_prob:2.455154914408922 - timing_s/adv:0.532898997887969 - timing_s/update_actor:15.235047852620482 - timing_s/step:30.448000945150852 - timing_s/stop_profile:0.00011887960135936737 - timing_per_token_ms/adv:0.003451306614992837 - timing_per_token_ms/gen:0.14734320057102976 - timing_per_token_ms/update_actor:0.09866939446663309 - perf/total_num_tokens:154405 - perf/time_per_step:30.448000945150852 - perf/throughput:633.8880846321642 (TaskRunner pid=2026458) Training Progress: 21%|██ | 21/100 [16:33<1:15:19, 57.20s/it] (TaskRunner pid=2026458) step:22 - global_seqlen/min:15143 - global_seqlen/max:18708 - global_seqlen/minmax_diff:3565 - global_seqlen/balanced_min:16965 - global_seqlen/balanced_max:16966 - global_seqlen/mean:16965.375 - actor/entropy:0.31038597226142883 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2530360817909241 - training/rollout_probs_diff_mean:0.0064188591204583645 - training/rollout_probs_diff_std:0.014548459090292454 - training/rollout_actor_probs_pearson_corr:0.9982183575630188 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0001835823059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.267485928721726e-05 - rollout_corr/rollout_is_max:1.9917649030685425 - rollout_corr/rollout_is_min:0.3914262652397156 - rollout_corr/rollout_is_std:0.0448860339820385 - rollout_corr/rollout_is_eff_sample_size:0.9979898885301821 - rollout_corr/rollout_is_seq_mean:1.0002237558364868 - rollout_corr/rollout_is_seq_std:0.0030402897391468287 - rollout_corr/rollout_is_seq_max:1.0082043409347534 - rollout_corr/rollout_is_seq_min:0.9909875988960266 - rollout_corr/rollout_is_seq_max_deviation:0.009012401103973389 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3948297398164868) - rollout_corr/training_log_ppl:np.float64(0.32873041287530214) - rollout_corr/kl:np.float64(0.0012066373147963105) - rollout_corr/k3_kl:np.float64(0.0013965993947522293) - rollout_corr/rollout_ppl:np.float64(1.3931263061240315) - rollout_corr/rollout_log_ppl:np.float64(0.32752377312863246) - rollout_corr/log_ppl_diff:np.float64(0.0012066397466696799) - rollout_corr/log_ppl_abs_diff:np.float64(0.002761501877103001) - rollout_corr/log_ppl_diff_max:np.float64(0.011306077241897583) - rollout_corr/log_ppl_diff_min:np.float64(-0.009509235620498657) - rollout_corr/ppl_ratio:np.float64(1.0012132688425481) - rollout_corr/chi2_token:np.float64(0.003218145342543721) - rollout_corr/chi2_seq:np.float64(1.0662864169571549) - actor/pg_loss:np.float64(0.006237241672351956) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020408303180374787) - actor/ppo_kl:np.float64(0.0003451099556370796) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(326844.6732725387) - self_distillation/token_reweight_w_std:np.float64(4298094.462517299) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025643010158092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1402466203435324) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.6192770302295685) - perf/mfu/actor:np.float64(0.03161979996245398) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.7820701599121) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003102818038314581 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003102818038314581 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.60546875 - response_length/max:646.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.60546875 - response_length_non_aborted/max:646.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012409500777721405 - timing_s/agent_loop/generate_sequences/min:np.float64(1.460020076483488) - timing_s/agent_loop/generate_sequences/max:np.float64(5.160831518471241) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.023891647448181) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.160831518471241) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:646 - timing_s/gen:10.945641493424773 - timing_s/reward:0.05562561750411987 - timing_s/old_log_prob:2.356655828654766 - timing_s/adv:0.4728220421820879 - timing_s/update_actor:15.27205784060061 - timing_s/step:29.190949207171798 - timing_s/stop_profile:0.0001829788088798523 - timing_per_token_ms/adv:0.003483728197741635 - timing_per_token_ms/gen:0.15570124032240534 - timing_per_token_ms/update_actor:0.11252372730193563 - perf/total_num_tokens:135723 - perf/time_per_step:29.190949207171798 - perf/throughput:581.1861368259944 (TaskRunner pid=2026458) Training Progress: 22%|██▏ | 22/100 [17:02<1:03:27, 48.81s/it] (TaskRunner pid=2026458) step:23 - global_seqlen/min:15206 - global_seqlen/max:25332 - global_seqlen/minmax_diff:10126 - global_seqlen/balanced_min:18411 - global_seqlen/balanced_max:25520 - global_seqlen/mean:19300.375 - actor/entropy:0.264564573764801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4770219326019287 - training/rollout_probs_diff_mean:0.0052871485240757465 - training/rollout_probs_diff_std:0.013520004227757454 - training/rollout_actor_probs_pearson_corr:0.998356282711029 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999374747276306 - rollout_corr/rollout_is_ratio_fraction_high:3.601138087105937e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0018966905772686e-05 - rollout_corr/rollout_is_max:2.1046066284179688 - rollout_corr/rollout_is_min:0.36882883310317993 - rollout_corr/rollout_is_std:0.041905514895915985 - rollout_corr/rollout_is_eff_sample_size:0.9982467687554272 - rollout_corr/rollout_is_seq_mean:0.9998911023139954 - rollout_corr/rollout_is_seq_std:0.0030532306991517544 - rollout_corr/rollout_is_seq_max:1.0095295906066895 - rollout_corr/rollout_is_seq_min:0.9908995628356934 - rollout_corr/rollout_is_seq_max_deviation:0.009529590606689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3668543784879148) - rollout_corr/training_log_ppl:np.float64(0.30833037884258374) - rollout_corr/kl:np.float64(0.0014712483040568713) - rollout_corr/k3_kl:np.float64(0.0014252641368557306) - rollout_corr/rollout_ppl:np.float64(1.364856572356075) - rollout_corr/rollout_log_ppl:np.float64(0.30685912866283616) - rollout_corr/log_ppl_diff:np.float64(0.0014712501797475852) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029776765513815917) - rollout_corr/log_ppl_diff_max:np.float64(0.01450282335281372) - rollout_corr/log_ppl_diff_min:np.float64(-0.008856803178787231) - rollout_corr/ppl_ratio:np.float64(1.0014787362888455) - rollout_corr/chi2_token:np.float64(0.003022435586899519) - rollout_corr/chi2_seq:np.float64(1.3533644466660917) - actor/pg_loss:np.float64(0.007422412978485227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020335467470431468) - actor/ppo_kl:np.float64(0.00035265562636155323) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(345985.51773530385) - self_distillation/token_reweight_w_std:np.float64(4731856.136310147) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988346283789724) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16046884056413546) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.7161731868982315) - perf/mfu/actor:np.float64(0.03389860046243484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1311912536621) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008347137831151485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008347137831151485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.41796875 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:325.41796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010981783270835876 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5618780367076397) - timing_s/agent_loop/generate_sequences/max:np.float64(49.69344619102776) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4613353605309385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.69344619102776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:230 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.576194578781724 - timing_s/reward:0.05657547526061535 - timing_s/old_log_prob:2.56242498382926 - timing_s/adv:0.5329509787261486 - timing_s/update_actor:16.533158656209707 - timing_s/step:75.34719325602055 - timing_s/stop_profile:0.00011706724762916565 - timing_per_token_ms/adv:0.0034516879770869 - timing_per_token_ms/gen:0.6671251464916721 - timing_per_token_ms/update_actor:0.1070779625798055 - perf/total_num_tokens:154403 - perf/time_per_step:75.34719325602055 - perf/throughput:256.1525408706292 (TaskRunner pid=2026458) Training Progress: 23%|██▎ | 23/100 [18:17<1:12:52, 56.79s/it] (TaskRunner pid=2026458) step:24 - global_seqlen/min:16231 - global_seqlen/max:19060 - global_seqlen/minmax_diff:2829 - global_seqlen/balanced_min:17704 - global_seqlen/balanced_max:17708 - global_seqlen/mean:17706.25 - actor/entropy:0.29770269989967346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5271099209785461 - training/rollout_probs_diff_mean:0.0061988383531570435 - training/rollout_probs_diff_std:0.014574181288480759 - training/rollout_actor_probs_pearson_corr:0.9981999397277832 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.000082015991211 - rollout_corr/rollout_is_ratio_fraction_high:2.8622949685086496e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011449179874034598 - rollout_corr/rollout_is_max:2.555720567703247 - rollout_corr/rollout_is_min:0.15318159759044647 - rollout_corr/rollout_is_std:0.04580353945493698 - rollout_corr/rollout_is_eff_sample_size:0.997906784084707 - rollout_corr/rollout_is_seq_mean:1.000064492225647 - rollout_corr/rollout_is_seq_std:0.0029369699768722057 - rollout_corr/rollout_is_seq_max:1.0110825300216675 - rollout_corr/rollout_is_seq_min:0.9918853640556335 - rollout_corr/rollout_is_seq_max_deviation:0.01108253002166748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36986547941342) - rollout_corr/training_log_ppl:np.float64(0.31092395351151936) - rollout_corr/kl:np.float64(0.0015822704315340275) - rollout_corr/k3_kl:np.float64(0.0015203862882344765) - rollout_corr/rollout_ppl:np.float64(1.3676151768304408) - rollout_corr/rollout_log_ppl:np.float64(0.3093416829360649) - rollout_corr/log_ppl_diff:np.float64(0.0015822705754544586) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029158030229154974) - rollout_corr/log_ppl_diff_max:np.float64(0.013341128826141357) - rollout_corr/log_ppl_diff_min:np.float64(-0.009624361991882324) - rollout_corr/ppl_ratio:np.float64(1.0015899618156254) - rollout_corr/chi2_token:np.float64(0.0028670839965343475) - rollout_corr/chi2_seq:np.float64(0.6717999817337841) - actor/pg_loss:np.float64(0.0065518481424078345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001698189102626202) - actor/ppo_kl:np.float64(0.000539918119613958) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(259653.62712082197) - self_distillation/token_reweight_w_std:np.float64(3592355.994360844) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997991549316794) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1357375134830363) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6166152656078339) - perf/mfu/actor:np.float64(0.03283040904184733) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1153450012207) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003980379085987806 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003980379085987806 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.9453125 - response_length/max:645.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.9453125 - response_length_non_aborted/max:645.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011000782251358032 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4089452624320984) - timing_s/agent_loop/generate_sequences/max:np.float64(5.263506431132555) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0320224383394816) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.263506431132555) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.14209609106183 - timing_s/reward:0.05296533182263374 - timing_s/old_log_prob:2.2817067373543978 - timing_s/adv:0.506876740604639 - timing_s/update_actor:15.04843096062541 - timing_s/step:29.098722390830517 - timing_s/stop_profile:0.00011877715587615967 - timing_per_token_ms/adv:0.003578374448320784 - timing_per_token_ms/gen:0.15945982899307082 - timing_per_token_ms/update_actor:0.10623671698288323 - perf/total_num_tokens:141650 - perf/time_per_step:29.098722390830517 - perf/throughput:608.4889144679262 (TaskRunner pid=2026458) Training Progress: 24%|██▍ | 24/100 [18:46<1:01:25, 48.49s/it] (TaskRunner pid=2026458) step:25 - global_seqlen/min:15795 - global_seqlen/max:22433 - global_seqlen/minmax_diff:6638 - global_seqlen/balanced_min:18523 - global_seqlen/balanced_max:18527 - global_seqlen/mean:18525.875 - actor/entropy:0.2758452296257019 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25533485412597656 - training/rollout_probs_diff_mean:0.005769852548837662 - training/rollout_probs_diff_std:0.013884530402719975 - training/rollout_actor_probs_pearson_corr:0.9983004331588745 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.78515625 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78515625 - rollout_corr/rollout_is_mean:1.0001115798950195 - rollout_corr/rollout_is_ratio_fraction_high:1.414367125107674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.828734250215348e-05 - rollout_corr/rollout_is_max:2.6026298999786377 - rollout_corr/rollout_is_min:0.3732539415359497 - rollout_corr/rollout_is_std:0.04336366429924965 - rollout_corr/rollout_is_eff_sample_size:0.9981233594863631 - rollout_corr/rollout_is_seq_mean:1.000120759010315 - rollout_corr/rollout_is_seq_std:0.003118946449831128 - rollout_corr/rollout_is_seq_max:1.0095044374465942 - rollout_corr/rollout_is_seq_min:0.990989625453949 - rollout_corr/rollout_is_seq_max_deviation:0.009504437446594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3535170801915228) - rollout_corr/training_log_ppl:np.float64(0.2983562831650488) - rollout_corr/kl:np.float64(0.000989430469498842) - rollout_corr/k3_kl:np.float64(0.0011073322453967194) - rollout_corr/rollout_ppl:np.float64(1.3521247510798275) - rollout_corr/rollout_log_ppl:np.float64(0.29736685164971277) - rollout_corr/log_ppl_diff:np.float64(0.0009894315153360367) - rollout_corr/log_ppl_abs_diff:np.float64(0.002447564620524645) - rollout_corr/log_ppl_diff_max:np.float64(0.01045176386833191) - rollout_corr/log_ppl_diff_min:np.float64(-0.008554220199584961) - rollout_corr/ppl_ratio:np.float64(1.0009945430792868) - rollout_corr/chi2_token:np.float64(0.002472628140822053) - rollout_corr/chi2_seq:np.float64(0.6732058441266418) - actor/pg_loss:np.float64(0.005739172571338713) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010865399485737726) - actor/ppo_kl:np.float64(0.0001121951680964628) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(393425.2932063611) - self_distillation/token_reweight_w_std:np.float64(5274741.3204780845) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018663464579731) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12906738638412207) - self_distillation/empty_target_batch:np.float64(0.21484375) - actor/grad_norm:np.float64(0.6542618125677109) - perf/mfu/actor:np.float64(0.034142301102186606) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1241111755371) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005874927621334791 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005874927621334791 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.18359375 - response_length/max:807.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.18359375 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014953315258026123 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3853862024843693) - timing_s/agent_loop/generate_sequences/max:np.float64(5.845100851729512) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.959879736532457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.845100851729512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.540503876283765 - timing_s/reward:0.05447870306670666 - timing_s/old_log_prob:2.3400896415114403 - timing_s/adv:0.5113725643604994 - timing_s/update_actor:15.447554072365165 - timing_s/step:29.938440999016166 - timing_s/stop_profile:0.00012017786502838135 - timing_per_token_ms/adv:0.0034503941403611123 - timing_per_token_ms/gen:0.16322509478075561 - timing_per_token_ms/update_actor:0.10422958478590866 - perf/total_num_tokens:148207 - perf/time_per_step:29.938440999016166 - perf/throughput:618.7989214471387 (TaskRunner pid=2026458) Training Progress: 25%|██▌ | 25/100 [19:16<53:40, 42.94s/it] (TaskRunner pid=2026458) step:26 - global_seqlen/min:16232 - global_seqlen/max:20091 - global_seqlen/minmax_diff:3859 - global_seqlen/balanced_min:18014 - global_seqlen/balanced_max:18018 - global_seqlen/mean:18016.5 - actor/entropy:0.28236204385757446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797455430030823 - training/rollout_probs_diff_mean:0.005817539524286985 - training/rollout_probs_diff_std:0.014095457270741463 - training/rollout_actor_probs_pearson_corr:0.9982751607894897 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000920295715332 - rollout_corr/rollout_is_ratio_fraction_high:1.4651584933744743e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.790950960246846e-05 - rollout_corr/rollout_is_max:2.0633111000061035 - rollout_corr/rollout_is_min:0.2806716561317444 - rollout_corr/rollout_is_std:0.044501952826976776 - rollout_corr/rollout_is_eff_sample_size:0.9980238466152811 - rollout_corr/rollout_is_seq_mean:1.0001037120819092 - rollout_corr/rollout_is_seq_std:0.0033811419270932674 - rollout_corr/rollout_is_seq_max:1.010282039642334 - rollout_corr/rollout_is_seq_min:0.9834394454956055 - rollout_corr/rollout_is_seq_max_deviation:0.01656055450439453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3481695111840963) - rollout_corr/training_log_ppl:np.float64(0.294715576863382) - rollout_corr/kl:np.float64(0.00122398861193318) - rollout_corr/k3_kl:np.float64(0.0011840371912512637) - rollout_corr/rollout_ppl:np.float64(1.3464941019192338) - rollout_corr/rollout_log_ppl:np.float64(0.2934915872756392) - rollout_corr/log_ppl_diff:np.float64(0.0012239895877428353) - rollout_corr/log_ppl_abs_diff:np.float64(0.002889484982006252) - rollout_corr/log_ppl_diff_max:np.float64(0.026334106922149658) - rollout_corr/log_ppl_diff_min:np.float64(-0.00859677791595459) - rollout_corr/ppl_ratio:np.float64(1.0012316501233727) - rollout_corr/chi2_token:np.float64(0.002297403523698449) - rollout_corr/chi2_seq:np.float64(1.2100839058402926) - actor/pg_loss:np.float64(0.004832616308704019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006622324299314641) - actor/ppo_kl:np.float64(0.00027824501646955824) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(57253.49309117114) - self_distillation/token_reweight_w_std:np.float64(780711.6470303052) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9960764090064913) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10869685839861631) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.5899917781352997) - perf/mfu/actor:np.float64(0.03340075709542584) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.18029022216797) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030969788786023855 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0030969788786023855 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:266.609375 - response_length/max:611.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.609375 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014692358672618866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2525565940886736) - timing_s/agent_loop/generate_sequences/max:np.float64(4.980492563918233) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.909446031990228) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.980492563918233) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:611 - timing_s/gen:10.703484870493412 - timing_s/reward:0.056105511263012886 - timing_s/old_log_prob:2.315383156761527 - timing_s/adv:0.49405837804079056 - timing_s/update_actor:15.426462525501847 - timing_s/step:29.082023203372955 - timing_s/stop_profile:0.0001284964382648468 - timing_per_token_ms/adv:0.003427818791391159 - timing_per_token_ms/gen:0.156823021603666 - timing_per_token_ms/update_actor:0.10703010105668309 - perf/total_num_tokens:144132 - perf/time_per_step:29.082023203372955 - perf/throughput:619.5064172120746 (TaskRunner pid=2026458) Training Progress: 26%|██▌ | 26/100 [19:46<47:51, 38.80s/it] (TaskRunner pid=2026458) step:27 - global_seqlen/min:14932 - global_seqlen/max:19899 - global_seqlen/minmax_diff:4967 - global_seqlen/balanced_min:17014 - global_seqlen/balanced_max:17065 - global_seqlen/mean:17022.125 - actor/entropy:0.27002352476119995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583366632461548 - training/rollout_probs_diff_mean:0.005672288592904806 - training/rollout_probs_diff_std:0.013904700987040997 - training/rollout_actor_probs_pearson_corr:0.9982785582542419 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:1.0000253915786743 - rollout_corr/rollout_is_ratio_fraction_high:5.697275264537893e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4243188161344733e-05 - rollout_corr/rollout_is_max:2.935737371444702 - rollout_corr/rollout_is_min:0.4851767420768738 - rollout_corr/rollout_is_std:0.04337328299880028 - rollout_corr/rollout_is_eff_sample_size:0.9981222906269569 - rollout_corr/rollout_is_seq_mean:0.9999536275863647 - rollout_corr/rollout_is_seq_std:0.002799020381644368 - rollout_corr/rollout_is_seq_max:1.0089412927627563 - rollout_corr/rollout_is_seq_min:0.991137683391571 - rollout_corr/rollout_is_seq_max_deviation:0.008941292762756348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3306819731369615) - rollout_corr/training_log_ppl:np.float64(0.28223902196623385) - rollout_corr/kl:np.float64(0.0010320211754617503) - rollout_corr/k3_kl:np.float64(0.0012100190330102123) - rollout_corr/rollout_ppl:np.float64(1.3292772802524269) - rollout_corr/rollout_log_ppl:np.float64(0.28120700089493766) - rollout_corr/log_ppl_diff:np.float64(0.0010320210712961853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025203333934769034) - rollout_corr/log_ppl_diff_max:np.float64(0.011266171932220459) - rollout_corr/log_ppl_diff_min:np.float64(-0.00728076696395874) - rollout_corr/ppl_ratio:np.float64(1.0010370928794146) - rollout_corr/chi2_token:np.float64(0.002755629364401102) - rollout_corr/chi2_seq:np.float64(0.8209603114519268) - actor/pg_loss:np.float64(0.0054146344773471355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001441872691430035) - actor/ppo_kl:np.float64(-3.021463783170475e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(149585.24241724727) - self_distillation/token_reweight_w_std:np.float64(2245182.637069527) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022315112873912) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10743028059368953) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.5845905467867851) - perf/mfu/actor:np.float64(0.031846668272466384) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.01116561889648) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008261048933491111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008261048933491111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.25390625 - response_length/max:877.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.25390625 - response_length_non_aborted/max:877.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001650247722864151 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4051890634000301) - timing_s/agent_loop/generate_sequences/max:np.float64(6.235670423135161) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9710693403030746) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.235670423135161) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:877 - timing_s/gen:12.108512347564101 - timing_s/reward:0.05487024039030075 - timing_s/old_log_prob:2.2881970033049583 - timing_s/adv:0.46778740733861923 - timing_s/update_actor:15.23558795452118 - timing_s/step:30.242075925692916 - timing_s/stop_profile:0.0001263841986656189 - timing_per_token_ms/adv:0.003435142552256396 - timing_per_token_ms/gen:0.17246382013081088 - timing_per_token_ms/update_actor:0.11188077248376142 - perf/total_num_tokens:136177 - perf/time_per_step:30.242075925692916 - perf/throughput:562.8623194328543 (TaskRunner pid=2026458) Training Progress: 27%|██▋ | 27/100 [20:16<44:06, 36.25s/it] (TaskRunner pid=2026458) step:28 - global_seqlen/min:15064 - global_seqlen/max:18912 - global_seqlen/minmax_diff:3848 - global_seqlen/balanced_min:17172 - global_seqlen/balanced_max:17302 - global_seqlen/mean:17191.25 - actor/entropy:0.26276373863220215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491259455680847 - training/rollout_probs_diff_mean:0.005732021760195494 - training/rollout_probs_diff_std:0.014186778105795383 - training/rollout_actor_probs_pearson_corr:0.9981420636177063 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999222159385681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4750567970622797e-05 - rollout_corr/rollout_is_max:1.8682423830032349 - rollout_corr/rollout_is_min:0.4175392985343933 - rollout_corr/rollout_is_std:0.043118301779031754 - rollout_corr/rollout_is_eff_sample_size:0.9981441433187315 - rollout_corr/rollout_is_seq_mean:0.9998924732208252 - rollout_corr/rollout_is_seq_std:0.0026770406402647495 - rollout_corr/rollout_is_seq_max:1.0065722465515137 - rollout_corr/rollout_is_seq_min:0.9909303188323975 - rollout_corr/rollout_is_seq_max_deviation:0.009069681167602539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3176847058348358) - rollout_corr/training_log_ppl:np.float64(0.2727040337776998) - rollout_corr/kl:np.float64(0.0012852028201848498) - rollout_corr/k3_kl:np.float64(0.001238845826378565) - rollout_corr/rollout_ppl:np.float64(1.315975757315755) - rollout_corr/rollout_log_ppl:np.float64(0.27141882907017134) - rollout_corr/log_ppl_diff:np.float64(0.0012852047075284645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026650355284800753) - rollout_corr/log_ppl_diff_max:np.float64(0.011441320180892944) - rollout_corr/log_ppl_diff_min:np.float64(-0.011876940727233887) - rollout_corr/ppl_ratio:np.float64(1.0012910023797303) - rollout_corr/chi2_token:np.float64(0.002501809736713767) - rollout_corr/chi2_seq:np.float64(1.4115001251921058) - actor/pg_loss:np.float64(0.005721935303881764) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001080054133581143) - actor/ppo_kl:np.float64(0.00020560735394870022) - actor/pg_clipfrac_lower:np.float64(3.0048076951061375e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(263679.60727620986) - self_distillation/token_reweight_w_std:np.float64(3103030.912776855) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007451251149178) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10346215419122018) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6124091595411301) - perf/mfu/actor:np.float64(0.03192732787611963) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.95586776733398) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00606801500543952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00606801500543952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:264.8203125 - response_length/max:1012.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:264.8203125 - response_length_non_aborted/max:1012.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001549404114484787 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3163175880908966) - timing_s/agent_loop/generate_sequences/max:np.float64(6.851215856149793) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.895652609608078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.851215856149793) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:255 - timing_s/agent_loop/slowest/response_length:1012 - timing_s/gen:12.757756907492876 - timing_s/reward:0.05424255505204201 - timing_s/old_log_prob:2.304752092808485 - timing_s/adv:0.5271155349910259 - timing_s/update_actor:15.178545750677586 - timing_s/step:30.909547282382846 - timing_s/stop_profile:0.00012030825018882751 - timing_per_token_ms/adv:0.0038327312949249324 - timing_per_token_ms/gen:0.18818415947565972 - timing_per_token_ms/update_actor:0.11036534392988864 - perf/total_num_tokens:137530 - perf/time_per_step:30.909547282382846 - perf/throughput:556.179288002652 (TaskRunner pid=2026458) Training Progress: 28%|██▊ | 28/100 [20:47<41:35, 34.66s/it] (TaskRunner pid=2026458) step:29 - global_seqlen/min:14445 - global_seqlen/max:22059 - global_seqlen/minmax_diff:7614 - global_seqlen/balanced_min:19100 - global_seqlen/balanced_max:20098 - global_seqlen/mean:19231.25 - actor/entropy:0.2399301379919052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5590871572494507 - training/rollout_probs_diff_mean:0.005236997734755278 - training/rollout_probs_diff_std:0.014018451794981956 - training/rollout_actor_probs_pearson_corr:0.9981542229652405 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.34375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.0000776052474976 - rollout_corr/rollout_is_ratio_fraction_high:2.4095805201795883e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.819161040359177e-05 - rollout_corr/rollout_is_max:3.816781997680664 - rollout_corr/rollout_is_min:0.17573890089988708 - rollout_corr/rollout_is_std:0.04291044920682907 - rollout_corr/rollout_is_eff_sample_size:0.9981623150356519 - rollout_corr/rollout_is_seq_mean:0.9999245405197144 - rollout_corr/rollout_is_seq_std:0.0027032301295548677 - rollout_corr/rollout_is_seq_max:1.008720874786377 - rollout_corr/rollout_is_seq_min:0.992788553237915 - rollout_corr/rollout_is_seq_max_deviation:0.008720874786376953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3173197288997471) - rollout_corr/training_log_ppl:np.float64(0.27068548381794244) - rollout_corr/kl:np.float64(0.0009471329129642214) - rollout_corr/k3_kl:np.float64(0.001118057121630045) - rollout_corr/rollout_ppl:np.float64(1.3160036713816226) - rollout_corr/rollout_log_ppl:np.float64(0.2697383483755402) - rollout_corr/log_ppl_diff:np.float64(0.0009471354424022138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024268297711387277) - rollout_corr/log_ppl_diff_max:np.float64(0.010676145553588867) - rollout_corr/log_ppl_diff_min:np.float64(-0.008322864770889282) - rollout_corr/ppl_ratio:np.float64(1.0009521523024887) - rollout_corr/chi2_token:np.float64(0.0025950868148356676) - rollout_corr/chi2_seq:np.float64(1.0168882303405553) - actor/pg_loss:np.float64(0.004738333867862821) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009898098714984371) - actor/ppo_kl:np.float64(-0.0001235502958607526) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34375) - self_distillation/token_reweight_w_mean:np.float64(148981.53039407637) - self_distillation/token_reweight_w_std:np.float64(2396815.2922347095) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019751288928092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.088682441302808) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5395381674170494) - perf/mfu/actor:np.float64(0.0350041185371077) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9663429260254) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014469530433416367 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014469530433416367 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.2265625 - response_length/max:2192.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.2265625 - response_length_non_aborted/max:2192.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000128256157040596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1863264609128237) - timing_s/agent_loop/generate_sequences/max:np.float64(12.72119532711804) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.387789061780495) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.72119532711804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:2192 - timing_s/gen:18.591661918908358 - timing_s/reward:0.05764943175017834 - timing_s/old_log_prob:2.3708045054227114 - timing_s/adv:0.6288922913372517 - timing_s/update_actor:15.632321024313569 - timing_s/step:37.36768543533981 - timing_s/stop_profile:0.00012124888598918915 - timing_per_token_ms/adv:0.004087697701249605 - timing_per_token_ms/gen:0.2239905293716821 - timing_per_token_ms/update_actor:0.10160754646937646 - perf/total_num_tokens:153850 - perf/time_per_step:37.36768543533981 - perf/throughput:514.6492156512428 (TaskRunner pid=2026458) Training Progress: 29%|██▉ | 29/100 [21:24<41:59, 35.49s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a key fragment: "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring with a chlorine substituent and a trifluoromethyl group. This fragment is likely derived from a reactant that contains a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a benzene ring, a chlorine substituent, and a carboxylic acid group, but it does not contain the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a benzene ring and a chlorine substituent, but it lacks the trifluoromethyl group and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a benzene ring, a chlorine substituent, and a trifluoromethyl group, and it matches the required fragment in the product. It also includes a carboxylic acid group, which is consistent with the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a benzene ring and a trifluoromethyl group, but it lacks the chlorine substituent and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, Option C is the only correct reactant that matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2026458) step:30 - global_seqlen/min:15398 - global_seqlen/max:27317 - global_seqlen/minmax_diff:11919 - global_seqlen/balanced_min:18528 - global_seqlen/balanced_max:25696 - global_seqlen/mean:20322.875 - actor/entropy:0.21429221332073212 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29036805033683777 - training/rollout_probs_diff_mean:0.0045964838936924934 - training/rollout_probs_diff_std:0.012965062633156776 - training/rollout_actor_probs_pearson_corr:0.9982911944389343 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:0.9999114871025085 - rollout_corr/rollout_is_ratio_fraction_high:1.198825157189276e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.99412560404744e-05 - rollout_corr/rollout_is_max:2.0840299129486084 - rollout_corr/rollout_is_min:0.03206612169742584 - rollout_corr/rollout_is_std:0.04015055671334267 - rollout_corr/rollout_is_eff_sample_size:0.9983904085296447 - rollout_corr/rollout_is_seq_mean:0.9999212622642517 - rollout_corr/rollout_is_seq_std:0.003086200449615717 - rollout_corr/rollout_is_seq_max:1.0128618478775024 - rollout_corr/rollout_is_seq_min:0.9903483986854553 - rollout_corr/rollout_is_seq_max_deviation:0.012861847877502441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3321797889657319) - rollout_corr/training_log_ppl:np.float64(0.28194430993880815) - rollout_corr/kl:np.float64(0.0011635422050204625) - rollout_corr/k3_kl:np.float64(0.0012278429346110897) - rollout_corr/rollout_ppl:np.float64(1.3305623242631555) - rollout_corr/rollout_log_ppl:np.float64(0.2807807675008007) - rollout_corr/log_ppl_diff:np.float64(0.0011635424380074255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028164471932541346) - rollout_corr/log_ppl_diff_max:np.float64(0.01170569658279419) - rollout_corr/log_ppl_diff_min:np.float64(-0.010286390781402588) - rollout_corr/ppl_ratio:np.float64(1.0011705257929862) - rollout_corr/chi2_token:np.float64(0.0025684768334031105) - rollout_corr/chi2_seq:np.float64(0.56981481006369) - actor/pg_loss:np.float64(0.004291482153348625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010325129969714908) - actor/ppo_kl:np.float64(2.040402626590776e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(97715.03854049812) - self_distillation/token_reweight_w_std:np.float64(1362849.7468459418) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998477473855019) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08768687405972742) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5216539651155472) - perf/mfu/actor:np.float64(0.03586380732978027) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9196662902832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/score/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/score/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/score/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07343617415290264) - val-core/sciknoweval/acc/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/acc/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10296365255394938) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7833380952380953) - val-core/sciknoweval/acc/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0895080232788665) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6183523809523809) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08896331489086151 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08896331489086151 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.83984375 - response_length/max:8192.0 - response_length/min:95.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:325.83984375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012153573334217072 - timing_s/agent_loop/generate_sequences/min:np.float64(0.691241292282939) - timing_s/agent_loop/generate_sequences/max:np.float64(49.47414601966739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9101952763448935) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.47414601966739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.4048219807446 - timing_s/reward:0.05735262483358383 - timing_s/old_log_prob:2.4137737192213535 - timing_s/adv:0.4771828204393387 - timing_s/update_actor:16.52942644432187 - timing_s/step:74.96756593696773 - timing_s/testing:65.94198310934007 - timing_s/save_checkpoint:6.872431704774499 - timing_s/stop_profile:9.614601731300354e-05 - timing_per_token_ms/adv:0.002935010551160568 - timing_per_token_ms/gen:0.6642069409667878 - timing_per_token_ms/update_actor:0.10166761865829681 - perf/total_num_tokens:162583 - perf/time_per_step:74.96756593696773 - perf/throughput:271.0889001930161 (TaskRunner pid=2026458) Training Progress: 30%|███ | 30/100 [23:52<1:20:43, 69.19s/it] (TaskRunner pid=2026458) step:31 - global_seqlen/min:15697 - global_seqlen/max:20497 - global_seqlen/minmax_diff:4800 - global_seqlen/balanced_min:18142 - global_seqlen/balanced_max:18440 - global_seqlen/mean:18179.75 - actor/entropy:0.24378487467765808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4056262969970703 - training/rollout_probs_diff_mean:0.005305483937263489 - training/rollout_probs_diff_std:0.013869675807654858 - training/rollout_actor_probs_pearson_corr:0.9981986880302429 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0002444982528687 - rollout_corr/rollout_is_ratio_fraction_high:5.420200977823697e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.485351620242e-05 - rollout_corr/rollout_is_max:2.6529436111450195 - rollout_corr/rollout_is_min:0.2698281407356262 - rollout_corr/rollout_is_std:0.04314593970775604 - rollout_corr/rollout_is_eff_sample_size:0.9981427181140134 - rollout_corr/rollout_is_seq_mean:1.0003163814544678 - rollout_corr/rollout_is_seq_std:0.0029239451978355646 - rollout_corr/rollout_is_seq_max:1.0140212774276733 - rollout_corr/rollout_is_seq_min:0.9939373135566711 - rollout_corr/rollout_is_seq_max_deviation:0.01402127742767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3019663002341986) - rollout_corr/training_log_ppl:np.float64(0.2597778052149806) - rollout_corr/kl:np.float64(0.001051263411028458) - rollout_corr/k3_kl:np.float64(0.0012008980110067569) - rollout_corr/rollout_ppl:np.float64(1.3005597270093858) - rollout_corr/rollout_log_ppl:np.float64(0.2587265418842435) - rollout_corr/log_ppl_diff:np.float64(0.0010512633307371289) - rollout_corr/log_ppl_abs_diff:np.float64(0.002555768267484382) - rollout_corr/log_ppl_diff_max:np.float64(0.010198384523391724) - rollout_corr/log_ppl_diff_min:np.float64(-0.010389238595962524) - rollout_corr/ppl_ratio:np.float64(1.00105693587102) - rollout_corr/chi2_token:np.float64(0.002714824862778187) - rollout_corr/chi2_seq:np.float64(1.5057584547903389) - actor/pg_loss:np.float64(0.004452571272850037) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820382315912866) - actor/ppo_kl:np.float64(0.00035900160938950876) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(99177.65471042739) - self_distillation/token_reweight_w_std:np.float64(1529254.068720376) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981596393045038) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09003465421847068) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5145964473485947) - perf/mfu/actor:np.float64(0.0335625674385555) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.96116638183594) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012466462794691324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012466462794691324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.2734375 - response_length/max:769.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.2734375 - response_length_non_aborted/max:769.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.449354648590088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2310489993542433) - timing_s/agent_loop/generate_sequences/max:np.float64(5.724285513162613) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1068192758539226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.724285513162613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:769 - timing_s/gen:11.310221493244171 - timing_s/reward:0.053344203159213066 - timing_s/old_log_prob:2.4446758944541216 - timing_s/adv:0.550066627562046 - timing_s/update_actor:15.255971036851406 - timing_s/step:29.709052259102464 - timing_s/stop_profile:0.0001292899250984192 - timing_per_token_ms/adv:0.003782138282718726 - timing_per_token_ms/gen:0.15325918714930176 - timing_per_token_ms/update_actor:0.10489673288171872 - perf/total_num_tokens:145438 - perf/time_per_step:29.709052259102464 - perf/throughput:611.9262856804853 (TaskRunner pid=2026458) Training Progress: 31%|███ | 31/100 [24:22<1:05:57, 57.36s/it] (TaskRunner pid=2026458) step:32 - global_seqlen/min:13741 - global_seqlen/max:20093 - global_seqlen/minmax_diff:6352 - global_seqlen/balanced_min:17020 - global_seqlen/balanced_max:17022 - global_seqlen/mean:17021.5 - actor/entropy:0.2510213851928711 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174631893634796 - training/rollout_probs_diff_mean:0.005626973696053028 - training/rollout_probs_diff_std:0.014290885999798775 - training/rollout_actor_probs_pearson_corr:0.9980513453483582 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0003660917282104 - rollout_corr/rollout_is_ratio_fraction_high:4.35514775745105e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.35514775745105e-05 - rollout_corr/rollout_is_max:2.588045358657837 - rollout_corr/rollout_is_min:0.49156951904296875 - rollout_corr/rollout_is_std:0.04363907128572464 - rollout_corr/rollout_is_eff_sample_size:0.9981006764057321 - rollout_corr/rollout_is_seq_mean:1.0004558563232422 - rollout_corr/rollout_is_seq_std:0.0026327052619308233 - rollout_corr/rollout_is_seq_max:1.0082557201385498 - rollout_corr/rollout_is_seq_min:0.9930630326271057 - rollout_corr/rollout_is_seq_max_deviation:0.008255720138549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3111304240301251) - rollout_corr/training_log_ppl:np.float64(0.2656050497171236) - rollout_corr/kl:np.float64(0.0011632751664905072) - rollout_corr/k3_kl:np.float64(0.0012394887605751137) - rollout_corr/rollout_ppl:np.float64(1.3095909063704312) - rollout_corr/rollout_log_ppl:np.float64(0.2644417754927417) - rollout_corr/log_ppl_diff:np.float64(0.0011632742243818939) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027760078373830765) - rollout_corr/log_ppl_diff_max:np.float64(0.027847379446029663) - rollout_corr/log_ppl_diff_min:np.float64(-0.00688682496547699) - rollout_corr/ppl_ratio:np.float64(1.0011707202065736) - rollout_corr/chi2_token:np.float64(0.00264995195902884) - rollout_corr/chi2_seq:np.float64(1.4071137343998998) - actor/pg_loss:np.float64(0.003960146103054285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013795204131383798) - actor/ppo_kl:np.float64(0.0005964390661219454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(250012.3793359641) - self_distillation/token_reweight_w_std:np.float64(2860040.7311532954) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007816257420927) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07969121757196262) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.522880308330059) - perf/mfu/actor:np.float64(0.031496540810405246) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.58367156982422) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019834069535136223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019834069535136223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.078125 - response_length/max:628.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.078125 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002002175897359848 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3522796910256147) - timing_s/agent_loop/generate_sequences/max:np.float64(4.971057282760739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.862827809440205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.971057282760739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.714683732017875 - timing_s/reward:0.05403785966336727 - timing_s/old_log_prob:2.382686661556363 - timing_s/adv:0.4786151312291622 - timing_s/update_actor:15.277625290676951 - timing_s/step:28.9955370221287 - timing_s/stop_profile:0.00011957064270973206 - timing_per_token_ms/adv:0.003514783738427593 - timing_per_token_ms/gen:0.1555467703968683 - timing_per_token_ms/update_actor:0.1121935881875639 - perf/total_num_tokens:136172 - perf/time_per_step:28.9955370221287 - perf/throughput:587.0386186332606 (TaskRunner pid=2026458) Training Progress: 32%|███▏ | 32/100 [24:51<55:22, 48.86s/it] (TaskRunner pid=2026458) step:33 - global_seqlen/min:15641 - global_seqlen/max:20620 - global_seqlen/minmax_diff:4979 - global_seqlen/balanced_min:18180 - global_seqlen/balanced_max:18182 - global_seqlen/mean:18181.0 - actor/entropy:0.28735455870628357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9959163069725037 - training/rollout_probs_diff_mean:0.00580747751519084 - training/rollout_probs_diff_std:0.015400327742099762 - training/rollout_actor_probs_pearson_corr:0.9980352520942688 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9997233152389526 - rollout_corr/rollout_is_ratio_fraction_high:1.423528101440752e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014235280104912817 - rollout_corr/rollout_is_max:2.051370859146118 - rollout_corr/rollout_is_min:0.0007158105727285147 - rollout_corr/rollout_is_std:0.04281798005104065 - rollout_corr/rollout_is_eff_sample_size:0.998169085060141 - rollout_corr/rollout_is_seq_mean:0.9997956156730652 - rollout_corr/rollout_is_seq_std:0.003073018742725253 - rollout_corr/rollout_is_seq_max:1.0114701986312866 - rollout_corr/rollout_is_seq_min:0.9903202056884766 - rollout_corr/rollout_is_seq_max_deviation:0.011470198631286621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3598110149614513) - rollout_corr/training_log_ppl:np.float64(0.30384603922721) - rollout_corr/kl:np.float64(0.0014454446327292203) - rollout_corr/k3_kl:np.float64(0.001423622587481077) - rollout_corr/rollout_ppl:np.float64(1.3577961758710444) - rollout_corr/rollout_log_ppl:np.float64(0.3024005940533243) - rollout_corr/log_ppl_diff:np.float64(0.0014454451738856733) - rollout_corr/log_ppl_abs_diff:np.float64(0.002951021713670343) - rollout_corr/log_ppl_diff_max:np.float64(0.02602332830429077) - rollout_corr/log_ppl_diff_min:np.float64(-0.010346651077270508) - rollout_corr/ppl_ratio:np.float64(1.001454961951822) - rollout_corr/chi2_token:np.float64(0.002324617700651288) - rollout_corr/chi2_seq:np.float64(0.6557224565185606) - actor/pg_loss:np.float64(0.0037985126255080104) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009114227987083723) - actor/ppo_kl:np.float64(7.80257138011109e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(186856.52135437424) - self_distillation/token_reweight_w_std:np.float64(2469605.76739274) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999388714786619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09890055964933708) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5402719005942345) - perf/mfu/actor:np.float64(0.03298787544887611) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.31770038604736) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003960967063903809 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003960967063903809 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.40625 - response_length/max:556.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.40625 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001493934541940689 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3925712388008833) - timing_s/agent_loop/generate_sequences/max:np.float64(4.813172671943903) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.01079661748372) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.813172671943903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.570713378489017 - timing_s/reward:0.056401949375867844 - timing_s/old_log_prob:2.26583200879395 - timing_s/adv:0.4962475970387459 - timing_s/update_actor:15.83989292383194 - timing_s/step:29.316704677417874 - timing_s/stop_profile:3.923475742340088e-05 - timing_per_token_ms/adv:0.00341185576315072 - timing_per_token_ms/gen:0.1504770723506579 - timing_per_token_ms/update_actor:0.10890416453874883 - perf/total_num_tokens:145448 - perf/time_per_step:29.316704677417874 - perf/throughput:620.1583772818947 (TaskRunner pid=2026458) Training Progress: 33%|███▎ | 33/100 [25:20<48:01, 43.01s/it] (TaskRunner pid=2026458) step:34 - global_seqlen/min:15048 - global_seqlen/max:22716 - global_seqlen/minmax_diff:7668 - global_seqlen/balanced_min:18406 - global_seqlen/balanced_max:18407 - global_seqlen/mean:18406.75 - actor/entropy:0.2822253406047821 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3892279863357544 - training/rollout_probs_diff_mean:0.00599341094493866 - training/rollout_probs_diff_std:0.014615834690630436 - training/rollout_actor_probs_pearson_corr:0.9981156587600708 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999809265136719 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012492018868215382 - rollout_corr/rollout_is_max:1.9517436027526855 - rollout_corr/rollout_is_min:0.10854519158601761 - rollout_corr/rollout_is_std:0.04540359228849411 - rollout_corr/rollout_is_eff_sample_size:0.9979426360193112 - rollout_corr/rollout_is_seq_mean:0.9999065399169922 - rollout_corr/rollout_is_seq_std:0.003091080579906702 - rollout_corr/rollout_is_seq_max:1.0100407600402832 - rollout_corr/rollout_is_seq_min:0.9879905581474304 - rollout_corr/rollout_is_seq_max_deviation:0.01200944185256958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3448055190965533) - rollout_corr/training_log_ppl:np.float64(0.29254834007588215) - rollout_corr/kl:np.float64(0.001122740535322464) - rollout_corr/k3_kl:np.float64(0.0011786174198959998) - rollout_corr/rollout_ppl:np.float64(1.3432867326773703) - rollout_corr/rollout_log_ppl:np.float64(0.2914255977375433) - rollout_corr/log_ppl_diff:np.float64(0.0011227423383388668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025718869001138955) - rollout_corr/log_ppl_diff_max:np.float64(0.014177501201629639) - rollout_corr/log_ppl_diff_min:np.float64(-0.006468132138252258) - rollout_corr/ppl_ratio:np.float64(1.0011283229105175) - rollout_corr/chi2_token:np.float64(0.002466450445353985) - rollout_corr/chi2_seq:np.float64(0.9479083714541048) - actor/pg_loss:np.float64(0.005841488018631935) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011660458308142552) - actor/ppo_kl:np.float64(-8.996075499190237e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(205396.02536904486) - self_distillation/token_reweight_w_std:np.float64(2610298.4065771694) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991585344541818) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12581025002873503) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.6120489835739136) - perf/mfu/actor:np.float64(0.03423697654860308) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.21736526489258) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012471199035644531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012471199035644531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.4296875 - response_length/max:743.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.4296875 - response_length_non_aborted/max:743.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.58306884765625e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4199188239872456) - timing_s/agent_loop/generate_sequences/max:np.float64(5.984945947304368) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2054700024527847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.984945947304368) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:207 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:11.809425836429 - timing_s/reward:0.05546233803033829 - timing_s/old_log_prob:2.252874342724681 - timing_s/adv:0.49194580502808094 - timing_s/update_actor:15.391067944467068 - timing_s/step:30.08788885548711 - timing_s/stop_profile:0.00011718645691871643 - timing_per_token_ms/adv:0.003340797567659153 - timing_per_token_ms/gen:0.16391507976055575 - timing_per_token_ms/update_actor:0.10452054235855779 - perf/total_num_tokens:147254 - perf/time_per_step:30.08788885548711 - perf/throughput:611.7660859626305 (TaskRunner pid=2026458) Training Progress: 34%|███▍ | 34/100 [25:50<43:03, 39.14s/it] (TaskRunner pid=2026458) step:35 - global_seqlen/min:14987 - global_seqlen/max:19906 - global_seqlen/minmax_diff:4919 - global_seqlen/balanced_min:16938 - global_seqlen/balanced_max:16939 - global_seqlen/mean:16938.25 - actor/entropy:0.294697642326355 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22079068422317505 - training/rollout_probs_diff_mean:0.005864571314305067 - training/rollout_probs_diff_std:0.013605127111077309 - training/rollout_actor_probs_pearson_corr:0.9984296560287476 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.00005042552948 - rollout_corr/rollout_is_ratio_fraction_high:3.1109037081478164e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.1109037081478164e-05 - rollout_corr/rollout_is_max:2.7191102504730225 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.04332791268825531 - rollout_corr/rollout_is_eff_sample_size:0.9981263285522803 - rollout_corr/rollout_is_seq_mean:1.0001163482666016 - rollout_corr/rollout_is_seq_std:0.0030649646650999784 - rollout_corr/rollout_is_seq_max:1.0084450244903564 - rollout_corr/rollout_is_seq_min:0.989280641078949 - rollout_corr/rollout_is_seq_max_deviation:0.010719358921051025 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3609930002130568) - rollout_corr/training_log_ppl:np.float64(0.3051015918608755) - rollout_corr/kl:np.float64(0.0011746971918284999) - rollout_corr/k3_kl:np.float64(0.0011032617354658214) - rollout_corr/rollout_ppl:np.float64(1.3593582352623343) - rollout_corr/rollout_log_ppl:np.float64(0.30392689420841634) - rollout_corr/log_ppl_diff:np.float64(0.0011746976524591446) - rollout_corr/log_ppl_abs_diff:np.float64(0.002850949880667031) - rollout_corr/log_ppl_diff_max:np.float64(0.012322753667831421) - rollout_corr/log_ppl_diff_min:np.float64(-0.010998740792274475) - rollout_corr/ppl_ratio:np.float64(1.0011811451986432) - rollout_corr/chi2_token:np.float64(0.0021009566262364388) - rollout_corr/chi2_seq:np.float64(1.5435644236858934) - actor/pg_loss:np.float64(0.0014815301401540637) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020538957323879004) - actor/ppo_kl:np.float64(0.00030627488916046275) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(94700.63521376462) - self_distillation/token_reweight_w_std:np.float64(1075744.331360741) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004025450907648) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03615615499438718) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.3742603175342083) - perf/mfu/actor:np.float64(0.031170659142369998) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1671371459961) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029106393922120333 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0029106393922120333 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:251.1328125 - response_length/max:502.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.1328125 - response_length_non_aborted/max:502.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001547783613204956 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1277420744299889) - timing_s/agent_loop/generate_sequences/max:np.float64(4.495345920324326) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.789818469223974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.495345920324326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:355 - timing_s/agent_loop/slowest/response_length:502 - timing_s/gen:10.292290568351746 - timing_s/reward:0.054091084748506546 - timing_s/old_log_prob:2.2543541602790356 - timing_s/adv:0.4955630674958229 - timing_s/update_actor:15.42586961016059 - timing_s/step:28.60832902789116 - timing_s/stop_profile:3.491714596748352e-05 - timing_per_token_ms/adv:0.003657130071700315 - timing_per_token_ms/gen:0.16009162495491905 - timing_per_token_ms/update_actor:0.11383901532153992 - perf/total_num_tokens:135506 - perf/time_per_step:28.60832902789116 - perf/throughput:592.0740768706333 (TaskRunner pid=2026458) Training Progress: 35%|███▌ | 35/100 [26:19<38:59, 35.99s/it] (TaskRunner pid=2026458) step:36 - global_seqlen/min:14431 - global_seqlen/max:20659 - global_seqlen/minmax_diff:6228 - global_seqlen/balanced_min:17408 - global_seqlen/balanced_max:17413 - global_seqlen/mean:17411.75 - actor/entropy:0.2818928360939026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30500367283821106 - training/rollout_probs_diff_mean:0.005986991338431835 - training/rollout_probs_diff_std:0.014329608529806137 - training/rollout_actor_probs_pearson_corr:0.9982293248176575 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000543594360352 - rollout_corr/rollout_is_ratio_fraction_high:1.4678683328384068e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.871473331353627e-05 - rollout_corr/rollout_is_max:3.0542242527008057 - rollout_corr/rollout_is_min:0.18286029994487762 - rollout_corr/rollout_is_std:0.044746387749910355 - rollout_corr/rollout_is_eff_sample_size:0.9980018804571298 - rollout_corr/rollout_is_seq_mean:1.0000989437103271 - rollout_corr/rollout_is_seq_std:0.0029557279776781797 - rollout_corr/rollout_is_seq_max:1.0128530263900757 - rollout_corr/rollout_is_seq_min:0.992638111114502 - rollout_corr/rollout_is_seq_max_deviation:0.012853026390075684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3384348950348794) - rollout_corr/training_log_ppl:np.float64(0.28769120943616144) - rollout_corr/kl:np.float64(0.0010638975770804748) - rollout_corr/k3_kl:np.float64(0.0012365762453327989) - rollout_corr/rollout_ppl:np.float64(1.3369859019294381) - rollout_corr/rollout_log_ppl:np.float64(0.28662730925134383) - rollout_corr/log_ppl_diff:np.float64(0.0010639001848176122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025838720030151308) - rollout_corr/log_ppl_diff_max:np.float64(0.01416441798210144) - rollout_corr/log_ppl_diff_min:np.float64(-0.009993910789489746) - rollout_corr/ppl_ratio:np.float64(1.0010696118697524) - rollout_corr/chi2_token:np.float64(0.002987496554851532) - rollout_corr/chi2_seq:np.float64(0.8903114418499172) - actor/pg_loss:np.float64(0.0051360129145905375) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000932457276576315) - actor/ppo_kl:np.float64(0.00010139281163645819) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(95758.95235729823) - self_distillation/token_reweight_w_std:np.float64(1392223.179759777) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9973206941504031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10905869025737047) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5497298240661621) - perf/mfu/actor:np.float64(0.03256102036135432) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1156120300293) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007596218958497047 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007596218958497047 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:266.1171875 - response_length/max:662.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.1171875 - response_length_non_aborted/max:662.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2092596888542175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3356867656111717) - timing_s/agent_loop/generate_sequences/max:np.float64(5.081710338592529) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.919884198134241) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.081710338592529) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:379 - timing_s/agent_loop/slowest/response_length:662 - timing_s/gen:10.837040143087506 - timing_s/reward:0.05424739792943001 - timing_s/old_log_prob:2.3121078684926033 - timing_s/adv:0.5094937365502119 - timing_s/update_actor:15.35438490100205 - timing_s/step:29.153722267597914 - timing_s/stop_profile:3.104284405708313e-05 - timing_per_token_ms/adv:0.0036576861641579098 - timing_per_token_ms/gen:0.1590734835905162 - timing_per_token_ms/update_actor:0.11023005227075143 - perf/total_num_tokens:139294 - perf/time_per_step:29.153722267597914 - perf/throughput:597.2393452945732 (TaskRunner pid=2026458) Training Progress: 36%|███▌ | 36/100 [26:48<36:12, 33.94s/it] (TaskRunner pid=2026458) step:37 - global_seqlen/min:15833 - global_seqlen/max:21178 - global_seqlen/minmax_diff:5345 - global_seqlen/balanced_min:18583 - global_seqlen/balanced_max:18587 - global_seqlen/mean:18585.625 - actor/entropy:0.285804808139801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4652300179004669 - training/rollout_probs_diff_mean:0.005965293850749731 - training/rollout_probs_diff_std:0.014233550056815147 - training/rollout_actor_probs_pearson_corr:0.9982559680938721 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0002163648605347 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9148995963623747e-05 - rollout_corr/rollout_is_max:1.695287823677063 - rollout_corr/rollout_is_min:0.16980920732021332 - rollout_corr/rollout_is_std:0.04377271980047226 - rollout_corr/rollout_is_eff_sample_size:0.9980885633504123 - rollout_corr/rollout_is_seq_mean:1.000288486480713 - rollout_corr/rollout_is_seq_std:0.0028211281169205904 - rollout_corr/rollout_is_seq_max:1.0087376832962036 - rollout_corr/rollout_is_seq_min:0.9919080138206482 - rollout_corr/rollout_is_seq_max_deviation:0.008737683296203613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3575056260451674) - rollout_corr/training_log_ppl:np.float64(0.3016296330606565) - rollout_corr/kl:np.float64(0.0007570147956474926) - rollout_corr/k3_kl:np.float64(0.0012164926273499077) - rollout_corr/rollout_ppl:np.float64(1.3564373790286481) - rollout_corr/rollout_log_ppl:np.float64(0.30087261548032984) - rollout_corr/log_ppl_diff:np.float64(0.0007570175803266466) - rollout_corr/log_ppl_abs_diff:np.float64(0.002560925087891519) - rollout_corr/log_ppl_diff_max:np.float64(0.011357903480529785) - rollout_corr/log_ppl_diff_min:np.float64(-0.008871495723724365) - rollout_corr/ppl_ratio:np.float64(1.000762677518651) - rollout_corr/chi2_token:np.float64(0.0033016859088093042) - rollout_corr/chi2_seq:np.float64(1.0105145464185625) - actor/pg_loss:np.float64(0.0031967939576134086) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006711441328661749) - actor/ppo_kl:np.float64(3.3348735911431504e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(214368.9940047427) - self_distillation/token_reweight_w_std:np.float64(2919033.6983316364) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007499558851123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07807663449784741) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4402663931250572) - perf/mfu/actor:np.float64(0.034623907048585915) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.02531814575195) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031590224243700504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031590224243700504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:268.01953125 - response_length/max:686.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:268.01953125 - response_length_non_aborted/max:686.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.923235297203064e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.457766680046916) - timing_s/agent_loop/generate_sequences/max:np.float64(5.311068296432495) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9142332015690044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.311068296432495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:686 - timing_s/gen:11.104413088411093 - timing_s/reward:0.05539845488965511 - timing_s/old_log_prob:2.2851091288030148 - timing_s/adv:0.5124162267893553 - timing_s/update_actor:15.311178967356682 - timing_s/step:29.35571401193738 - timing_s/stop_profile:0.00011339224874973297 - timing_per_token_ms/adv:0.003446320925374821 - timing_per_token_ms/gen:0.1618412412867983 - timing_per_token_ms/update_actor:0.10297729406030656 - perf/total_num_tokens:148685 - perf/time_per_step:29.35571401193738 - perf/throughput:633.1177975246056 (TaskRunner pid=2026458) Training Progress: 37%|███▋ | 37/100 [27:18<34:12, 32.58s/it] (TaskRunner pid=2026458) step:38 - global_seqlen/min:15292 - global_seqlen/max:19759 - global_seqlen/minmax_diff:4467 - global_seqlen/balanced_min:17036 - global_seqlen/balanced_max:17040 - global_seqlen/mean:17038.625 - actor/entropy:0.2831912040710449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4492228925228119 - training/rollout_probs_diff_mean:0.006005707662552595 - training/rollout_probs_diff_std:0.014447803609073162 - training/rollout_actor_probs_pearson_corr:0.9981924891471863 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998399019241333 - rollout_corr/rollout_is_ratio_fraction_high:2.9010312573518604e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3515468860277906e-05 - rollout_corr/rollout_is_max:4.976550579071045 - rollout_corr/rollout_is_min:0.1723451316356659 - rollout_corr/rollout_is_std:0.045169975608587265 - rollout_corr/rollout_is_eff_sample_size:0.997963056153362 - rollout_corr/rollout_is_seq_mean:0.999801754951477 - rollout_corr/rollout_is_seq_std:0.0029568150639533997 - rollout_corr/rollout_is_seq_max:1.01389741897583 - rollout_corr/rollout_is_seq_min:0.9917296171188354 - rollout_corr/rollout_is_seq_max_deviation:0.013897418975830078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3477132506668568) - rollout_corr/training_log_ppl:np.float64(0.29560438107000664) - rollout_corr/kl:np.float64(0.0015795158470695014) - rollout_corr/k3_kl:np.float64(0.0011915537743334426) - rollout_corr/rollout_ppl:np.float64(1.3455825243145227) - rollout_corr/rollout_log_ppl:np.float64(0.29402486264007166) - rollout_corr/log_ppl_diff:np.float64(0.0015795184299349785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002752322005107999) - rollout_corr/log_ppl_diff_max:np.float64(0.010404467582702637) - rollout_corr/log_ppl_diff_min:np.float64(-0.006978675723075867) - rollout_corr/ppl_ratio:np.float64(1.0015854325611144) - rollout_corr/chi2_token:np.float64(0.0016510849818587303) - rollout_corr/chi2_seq:np.float64(0.42648625490255654) - actor/pg_loss:np.float64(0.004350315313786268) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001112287538489909) - actor/ppo_kl:np.float64(0.0002975818272203945) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(277881.75766049954) - self_distillation/token_reweight_w_std:np.float64(3405953.521727615) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997533587738872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09852648590458557) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5438188463449478) - perf/mfu/actor:np.float64(0.03193194428398073) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.85803985595703) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00237884558737278 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00237884558737278 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.30078125 - response_length/max:549.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.30078125 - response_length_non_aborted/max:549.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013971328735351562 - timing_s/agent_loop/generate_sequences/min:np.float64(1.483020644634962) - timing_s/agent_loop/generate_sequences/max:np.float64(4.714750545099378) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0552332281222334) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.714750545099378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:549 - timing_s/gen:10.464355397969484 - timing_s/reward:0.05412936769425869 - timing_s/old_log_prob:2.2855528369545937 - timing_s/adv:0.4730773251503706 - timing_s/update_actor:15.16877150721848 - timing_s/step:28.533766953274608 - timing_s/stop_profile:0.00012415647506713867 - timing_per_token_ms/adv:0.003470624281231398 - timing_per_token_ms/gen:0.15178711358943855 - timing_per_token_ms/update_actor:0.11128224480568767 - perf/total_num_tokens:136309 - perf/time_per_step:28.533766953274608 - perf/throughput:597.1389977321099 (TaskRunner pid=2026458) Training Progress: 38%|███▊ | 38/100 [27:46<32:25, 31.38s/it] (TaskRunner pid=2026458) step:39 - global_seqlen/min:15017 - global_seqlen/max:21775 - global_seqlen/minmax_diff:6758 - global_seqlen/balanced_min:17748 - global_seqlen/balanced_max:17751 - global_seqlen/mean:17750.375 - actor/entropy:0.2934376001358032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.006076316349208355 - training/rollout_probs_diff_std:0.014462748542428017 - training/rollout_actor_probs_pearson_corr:0.9982326030731201 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001943111419678 - rollout_corr/rollout_is_ratio_fraction_high:2.8678357921307907e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.735671584261581e-05 - rollout_corr/rollout_is_max:2.3144755363464355 - rollout_corr/rollout_is_min:0.44314810633659363 - rollout_corr/rollout_is_std:0.04461565613746643 - rollout_corr/rollout_is_eff_sample_size:0.9980141101447896 - rollout_corr/rollout_is_seq_mean:1.0003681182861328 - rollout_corr/rollout_is_seq_std:0.0030234723817557096 - rollout_corr/rollout_is_seq_max:1.012935996055603 - rollout_corr/rollout_is_seq_min:0.991753876209259 - rollout_corr/rollout_is_seq_max_deviation:0.012935996055603027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3561590211465955) - rollout_corr/training_log_ppl:np.float64(0.30050787163781933) - rollout_corr/kl:np.float64(0.0008042539968293738) - rollout_corr/k3_kl:np.float64(0.0011359995904740572) - rollout_corr/rollout_ppl:np.float64(1.355069079902023) - rollout_corr/rollout_log_ppl:np.float64(0.2997036176966503) - rollout_corr/log_ppl_diff:np.float64(0.0008042539411690086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024447946052532643) - rollout_corr/log_ppl_diff_max:np.float64(0.010187238454818726) - rollout_corr/log_ppl_diff_min:np.float64(-0.010493099689483643) - rollout_corr/ppl_ratio:np.float64(1.0008093006908894) - rollout_corr/chi2_token:np.float64(0.0029845864046365023) - rollout_corr/chi2_seq:np.float64(0.9699735399335623) - actor/pg_loss:np.float64(0.003664600197225809) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006868128334645007) - actor/ppo_kl:np.float64(0.00012619273251779362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(231536.99509470444) - self_distillation/token_reweight_w_std:np.float64(3103805.8869296443) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012708925642073) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08035396633204073) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.45319733768701553) - perf/mfu/actor:np.float64(0.03298738163573484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.00569915771484) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005079654045403004 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005079654045403004 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.41796875 - response_length/max:558.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.41796875 - response_length_non_aborted/max:558.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.198114275932312e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3789098169654608) - timing_s/agent_loop/generate_sequences/max:np.float64(4.869980722665787) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.020734860765515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.869980722665787) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:365 - timing_s/agent_loop/slowest/response_length:558 - timing_s/gen:10.739610413089395 - timing_s/reward:0.054600564762949944 - timing_s/old_log_prob:2.247015619650483 - timing_s/adv:0.5021204594522715 - timing_s/update_actor:15.248395329341292 - timing_s/step:28.878361692652106 - timing_s/stop_profile:0.00011930800974369049 - timing_per_token_ms/adv:0.0035359848697018477 - timing_per_token_ms/gen:0.15399719544429077 - timing_per_token_ms/update_actor:0.10738079709119731 - perf/total_num_tokens:142003 - perf/time_per_step:28.878361692652106 - perf/throughput:614.6600416226678 (TaskRunner pid=2026458) Training Progress: 39%|███▉ | 39/100 [28:15<31:09, 30.65s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a nitrogen atom bonded to a carbon chain, with a substituted aromatic ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. The nitrogen is bonded to a substituted aromatic ring with a chlorine atom and a trifluoromethyl group, matching the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it lacks the specific chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes an oxygen group (Oc2) instead of the chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes a double bond (CC=C) and a different trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Based on the structural analysis, Option A is the only one that matches the product's structure, making it the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2026458) step:40 - global_seqlen/min:16589 - global_seqlen/max:23076 - global_seqlen/minmax_diff:6487 - global_seqlen/balanced_min:19038 - global_seqlen/balanced_max:19042 - global_seqlen/mean:19040.5 - actor/entropy:0.3083128035068512 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48448076844215393 - training/rollout_probs_diff_mean:0.006092434749007225 - training/rollout_probs_diff_std:0.01408663671463728 - training/rollout_actor_probs_pearson_corr:0.9983260631561279 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0004887580871582 - rollout_corr/rollout_is_ratio_fraction_high:2.767170371953398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.383585185976699e-05 - rollout_corr/rollout_is_max:2.5501346588134766 - rollout_corr/rollout_is_min:0.2322635054588318 - rollout_corr/rollout_is_std:0.044838204979896545 - rollout_corr/rollout_is_eff_sample_size:0.9979954688959803 - rollout_corr/rollout_is_seq_mean:1.0004291534423828 - rollout_corr/rollout_is_seq_std:0.00280501926317811 - rollout_corr/rollout_is_seq_max:1.0100523233413696 - rollout_corr/rollout_is_seq_min:0.9925862550735474 - rollout_corr/rollout_is_seq_max_deviation:0.010052323341369629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3748237513937056) - rollout_corr/training_log_ppl:np.float64(0.3138403666380327) - rollout_corr/kl:np.float64(0.0008364543262393553) - rollout_corr/k3_kl:np.float64(0.0011391592736345046) - rollout_corr/rollout_ppl:np.float64(1.3736085868440568) - rollout_corr/rollout_log_ppl:np.float64(0.3130039106181357) - rollout_corr/log_ppl_diff:np.float64(0.0008364560198970139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023778348113410175) - rollout_corr/log_ppl_diff_max:np.float64(0.009519308805465698) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073702335357666016) - rollout_corr/ppl_ratio:np.float64(1.0008412639144808) - rollout_corr/chi2_token:np.float64(0.0029417627956718206) - rollout_corr/chi2_seq:np.float64(0.8329082692507654) - actor/pg_loss:np.float64(0.0021506232442334294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047293603620346403) - actor/ppo_kl:np.float64(0.0002316894797989022) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(45010.86430421425) - self_distillation/token_reweight_w_std:np.float64(529273.9801185789) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998293393291533) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05143398354994133) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.38673160597682) - perf/mfu/actor:np.float64(0.03526754309862706) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.86711502075195) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/reward/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/score/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/score/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/score/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06656845162631347) - val-core/sciknoweval/acc/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/acc/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09064171674985216) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7905285714285714) - val-core/sciknoweval/acc/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07673490521638164) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6577380952380952) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002604287412166728) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015362291495737217) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997095238095238) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011396681698700574) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989476190476191) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.001975811782419187) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999333333333333) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005594782149609339) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980619047619048) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023394041530661654) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00015050933932646775) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9969285714285715) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002278634869993062) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034849049989134073 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034849049989134073 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:282.328125 - response_length/max:680.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:282.328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018454715609550476 - timing_s/agent_loop/generate_sequences/min:np.float64(1.287922067567706) - timing_s/agent_loop/generate_sequences/max:np.float64(5.481446422636509) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0874510004068725) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.481446422636509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.202703416347504 - timing_s/reward:0.05725381709635258 - timing_s/old_log_prob:2.3498482443392277 - timing_s/adv:0.48736315965652466 - timing_s/update_actor:15.279724219813943 - timing_s/step:29.463446816429496 - timing_s/testing:112.37399195507169 - timing_s/save_checkpoint:6.7144703436642885 - timing_s/stop_profile:0.00011608190834522247 - timing_per_token_ms/adv:0.003199516554558209 - timing_per_token_ms/gen:0.15499894039995993 - timing_per_token_ms/update_actor:0.10031068130966848 - perf/total_num_tokens:152324 - perf/time_per_step:29.463446816429496 - perf/throughput:646.2414298853378 (TaskRunner pid=2026458) Training Progress: 40%|████ | 40/100 [30:44<1:06:02, 66.04s/it] (TaskRunner pid=2026458) step:41 - global_seqlen/min:14161 - global_seqlen/max:22430 - global_seqlen/minmax_diff:8269 - global_seqlen/balanced_min:17529 - global_seqlen/balanced_max:17569 - global_seqlen/mean:17540.0 - actor/entropy:0.28421929478645325 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3167548179626465 - training/rollout_probs_diff_mean:0.005681131035089493 - training/rollout_probs_diff_std:0.013473561964929104 - training/rollout_actor_probs_pearson_corr:0.9984502792358398 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0002758502960205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.967482328414917 - rollout_corr/rollout_is_min:0.5226783752441406 - rollout_corr/rollout_is_std:0.04252113774418831 - rollout_corr/rollout_is_eff_sample_size:0.998196166076475 - rollout_corr/rollout_is_seq_mean:1.000309944152832 - rollout_corr/rollout_is_seq_std:0.002816120395436883 - rollout_corr/rollout_is_seq_max:1.0100390911102295 - rollout_corr/rollout_is_seq_min:0.9916365146636963 - rollout_corr/rollout_is_seq_max_deviation:0.010039091110229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3460121774114668) - rollout_corr/training_log_ppl:np.float64(0.29434393605333753) - rollout_corr/kl:np.float64(0.0006344841518632904) - rollout_corr/k3_kl:np.float64(0.0010341189710061371) - rollout_corr/rollout_ppl:np.float64(1.3451492232270539) - rollout_corr/rollout_log_ppl:np.float64(0.2937094499648083) - rollout_corr/log_ppl_diff:np.float64(0.0006344860885292292) - rollout_corr/log_ppl_abs_diff:np.float64(0.002157465787604451) - rollout_corr/log_ppl_diff_max:np.float64(0.00896558165550232) - rollout_corr/log_ppl_diff_min:np.float64(-0.008629709482192993) - rollout_corr/ppl_ratio:np.float64(1.000638237921521) - rollout_corr/chi2_token:np.float64(0.0028806456830352545) - rollout_corr/chi2_seq:np.float64(0.6779269557446241) - actor/pg_loss:np.float64(0.004367844783701003) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004992729859623068) - actor/ppo_kl:np.float64(5.57020750591164e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(156417.83956303308) - self_distillation/token_reweight_w_std:np.float64(2357416.5479677515) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000363286351785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0830252695013769) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.500577874481678) - perf/mfu/actor:np.float64(0.032587394156940674) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.82418060302734) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022402419708669186 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022402419708669186 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.21875 - response_length/max:788.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.21875 - response_length_non_aborted/max:788.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017307698726654053 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2802286799997091) - timing_s/agent_loop/generate_sequences/max:np.float64(5.627637119963765) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.104791782410757) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.627637119963765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:788 - timing_s/gen:11.781217468902469 - timing_s/reward:0.05423154681921005 - timing_s/old_log_prob:2.465151973068714 - timing_s/adv:0.5479183085262775 - timing_s/update_actor:15.487478408962488 - timing_s/step:30.435431083664298 - timing_s/stop_profile:0.00011926889419555664 - timing_per_token_ms/adv:0.0039047769991895492 - timing_per_token_ms/gen:0.16191887670289265 - timing_per_token_ms/update_actor:0.11037256562829595 - perf/total_num_tokens:140320 - perf/time_per_step:30.435431083664298 - perf/throughput:576.3020064274463 (TaskRunner pid=2026458) Training Progress: 41%|████ | 41/100 [31:14<54:26, 55.37s/it] (TaskRunner pid=2026458) step:42 - global_seqlen/min:15278 - global_seqlen/max:20664 - global_seqlen/minmax_diff:5386 - global_seqlen/balanced_min:18018 - global_seqlen/balanced_max:18114 - global_seqlen/mean:18031.0 - actor/entropy:0.2842845022678375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.289776086807251 - training/rollout_probs_diff_mean:0.005445326212793589 - training/rollout_probs_diff_std:0.013068844564259052 - training/rollout_actor_probs_pearson_corr:0.9985550045967102 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997981786727905 - rollout_corr/rollout_is_ratio_fraction_high:1.2730420166917611e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5460840333835222e-05 - rollout_corr/rollout_is_max:2.036485433578491 - rollout_corr/rollout_is_min:0.43032172322273254 - rollout_corr/rollout_is_std:0.04114258661866188 - rollout_corr/rollout_is_eff_sample_size:0.998309494689457 - rollout_corr/rollout_is_seq_mean:0.999789297580719 - rollout_corr/rollout_is_seq_std:0.0025704929139465094 - rollout_corr/rollout_is_seq_max:1.008382797241211 - rollout_corr/rollout_is_seq_min:0.9924810528755188 - rollout_corr/rollout_is_seq_max_deviation:0.008382797241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3495313432067633) - rollout_corr/training_log_ppl:np.float64(0.2967612352222204) - rollout_corr/kl:np.float64(0.0012985346789022856) - rollout_corr/k3_kl:np.float64(0.0010939320847001) - rollout_corr/rollout_ppl:np.float64(1.3477259748615324) - rollout_corr/rollout_log_ppl:np.float64(0.29546269873389974) - rollout_corr/log_ppl_diff:np.float64(0.0012985364883206785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002426193736027926) - rollout_corr/log_ppl_diff_max:np.float64(0.00859573483467102) - rollout_corr/log_ppl_diff_min:np.float64(-0.006498366594314575) - rollout_corr/ppl_ratio:np.float64(1.0013033242430538) - rollout_corr/chi2_token:np.float64(0.001749418443068862) - rollout_corr/chi2_seq:np.float64(0.668122704140842) - actor/pg_loss:np.float64(0.0077214669436216354) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001590086207670538) - actor/ppo_kl:np.float64(0.0002024844293835315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(176662.73121298593) - self_distillation/token_reweight_w_std:np.float64(2621435.3882032083) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981021112762392) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14360627194400877) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.6128042787313461) - perf/mfu/actor:np.float64(0.03360615388418303) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.27114486694336) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003445170121267438 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003445170121267438 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:306.84375 - response_length/max:965.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:306.84375 - response_length_non_aborted/max:965.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013544969260692596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3969923835247755) - timing_s/agent_loop/generate_sequences/max:np.float64(6.683027235791087) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.339469947764883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.683027235791087) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:965 - timing_s/gen:13.290308959782124 - timing_s/reward:0.05642969720065594 - timing_s/old_log_prob:2.2229634914547205 - timing_s/adv:0.47765903919935226 - timing_s/update_actor:15.349972175434232 - timing_s/step:31.484555166214705 - timing_s/stop_profile:3.08305025100708e-05 - timing_per_token_ms/adv:0.003311373739666077 - timing_per_token_ms/gen:0.16919122313603885 - timing_per_token_ms/update_actor:0.10641376085238084 - perf/total_num_tokens:144248 - perf/time_per_step:31.484555166214705 - perf/throughput:572.6934970117862 (TaskRunner pid=2026458) Training Progress: 42%|████▏ | 42/100 [31:46<46:36, 48.21s/it] (TaskRunner pid=2026458) step:43 - global_seqlen/min:17746 - global_seqlen/max:21005 - global_seqlen/minmax_diff:3259 - global_seqlen/balanced_min:19023 - global_seqlen/balanced_max:19027 - global_seqlen/mean:19025.375 - actor/entropy:0.29123467206954956 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.277299165725708 - training/rollout_probs_diff_mean:0.0057848598808050156 - training/rollout_probs_diff_std:0.013805463910102844 - training/rollout_actor_probs_pearson_corr:0.9984029531478882 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998417496681213 - rollout_corr/rollout_is_ratio_fraction_high:1.3395668247540016e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.697834032820538e-05 - rollout_corr/rollout_is_max:2.0824544429779053 - rollout_corr/rollout_is_min:0.3074188232421875 - rollout_corr/rollout_is_std:0.0433402918279171 - rollout_corr/rollout_is_eff_sample_size:0.9981245471106103 - rollout_corr/rollout_is_seq_mean:0.9999098181724548 - rollout_corr/rollout_is_seq_std:0.0025561333168298006 - rollout_corr/rollout_is_seq_max:1.0072154998779297 - rollout_corr/rollout_is_seq_min:0.9910159111022949 - rollout_corr/rollout_is_seq_max_deviation:0.008984088897705078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3661411092616618) - rollout_corr/training_log_ppl:np.float64(0.30844287818763405) - rollout_corr/kl:np.float64(0.001555773359804391) - rollout_corr/k3_kl:np.float64(0.0014595502659631165) - rollout_corr/rollout_ppl:np.float64(1.3639800301752985) - rollout_corr/rollout_log_ppl:np.float64(0.30688710472895764) - rollout_corr/log_ppl_diff:np.float64(0.0015557734586764127) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026391229184810072) - rollout_corr/log_ppl_diff_max:np.float64(0.011147260665893555) - rollout_corr/log_ppl_diff_min:np.float64(-0.006583303213119507) - rollout_corr/ppl_ratio:np.float64(1.0015615709125996) - rollout_corr/chi2_token:np.float64(0.0026983728166669607) - rollout_corr/chi2_seq:np.float64(0.5271497289650142) - actor/pg_loss:np.float64(0.0038334736600518227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013588511892521637) - actor/ppo_kl:np.float64(0.00048035153239256445) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(120698.68936810177) - self_distillation/token_reweight_w_std:np.float64(1773706.5877801427) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000403418438509) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07702912960667163) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.46645358949899673) - perf/mfu/actor:np.float64(0.03541705134193752) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77776336669922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00016074800805654377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00016074800805654377 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.60546875 - response_length/max:660.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.60546875 - response_length_non_aborted/max:660.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.972890019416809e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.429830538108945) - timing_s/agent_loop/generate_sequences/max:np.float64(5.387127343565226) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1587736944129574) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.387127343565226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:660 - timing_s/gen:11.481718624010682 - timing_s/reward:0.056908562779426575 - timing_s/old_log_prob:2.2525762263685465 - timing_s/adv:0.5500077605247498 - timing_s/update_actor:15.172130836173892 - timing_s/step:29.60062885284424 - timing_s/stop_profile:0.00011467933654785156 - timing_per_token_ms/adv:0.0036136459894006673 - timing_per_token_ms/gen:0.15380528893130277 - timing_per_token_ms/update_actor:0.09968352027341046 - perf/total_num_tokens:152203 - perf/time_per_step:29.60062885284424 - perf/throughput:642.7355004713661 (TaskRunner pid=2026458) Training Progress: 43%|████▎ | 43/100 [32:15<40:30, 42.64s/it] (TaskRunner pid=2026458) step:44 - global_seqlen/min:16682 - global_seqlen/max:21465 - global_seqlen/minmax_diff:4783 - global_seqlen/balanced_min:19287 - global_seqlen/balanced_max:19289 - global_seqlen/mean:19287.75 - actor/entropy:0.2704760432243347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26803696155548096 - training/rollout_probs_diff_mean:0.005588939879089594 - training/rollout_probs_diff_std:0.013731644488871098 - training/rollout_actor_probs_pearson_corr:0.9983340501785278 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000927448272705 - rollout_corr/rollout_is_ratio_fraction_high:2.68305120698642e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.049153257161379e-05 - rollout_corr/rollout_is_max:2.0968687534332275 - rollout_corr/rollout_is_min:0.22089464962482452 - rollout_corr/rollout_is_std:0.04294377565383911 - rollout_corr/rollout_is_eff_sample_size:0.9981595832973885 - rollout_corr/rollout_is_seq_mean:1.0000972747802734 - rollout_corr/rollout_is_seq_std:0.00284771085716784 - rollout_corr/rollout_is_seq_max:1.0088731050491333 - rollout_corr/rollout_is_seq_min:0.9906178116798401 - rollout_corr/rollout_is_seq_max_deviation:0.009382188320159912 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3369123949669302) - rollout_corr/training_log_ppl:np.float64(0.2868449309025891) - rollout_corr/kl:np.float64(0.0009355866875218055) - rollout_corr/k3_kl:np.float64(0.0010745994904937106) - rollout_corr/rollout_ppl:np.float64(1.335645436309278) - rollout_corr/rollout_log_ppl:np.float64(0.28590934313251637) - rollout_corr/log_ppl_diff:np.float64(0.0009355877700727433) - rollout_corr/log_ppl_abs_diff:np.float64(0.002383651997661218) - rollout_corr/log_ppl_diff_max:np.float64(0.011576682329177856) - rollout_corr/log_ppl_diff_min:np.float64(-0.006330162286758423) - rollout_corr/ppl_ratio:np.float64(1.00094054476358) - rollout_corr/chi2_token:np.float64(0.00241070962511003) - rollout_corr/chi2_seq:np.float64(0.8048715838231146) - actor/pg_loss:np.float64(0.0025291290367022157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006139230463304557) - actor/ppo_kl:np.float64(5.7461318307527876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(186553.62624621042) - self_distillation/token_reweight_w_std:np.float64(2401896.473150722) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013129995204508) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05587577508413233) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.40642886608839035) - perf/mfu/actor:np.float64(0.03559055533852201) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.5995101928711) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029178180266171694 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029178180266171694 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1796875 - response_length/max:674.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1796875 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001512337476015091 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5085251331329346) - timing_s/agent_loop/generate_sequences/max:np.float64(5.416358392685652) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1938710649119457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.416358392685652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:634 - timing_s/gen:11.832684468477964 - timing_s/reward:0.057033905759453773 - timing_s/old_log_prob:2.2766003496944904 - timing_s/adv:0.5443154312670231 - timing_s/update_actor:15.36143240518868 - timing_s/step:30.159771636128426 - timing_s/stop_profile:0.0001326482743024826 - timing_per_token_ms/adv:0.003527598030271954 - timing_per_token_ms/gen:0.1587384892876226 - timing_per_token_ms/update_actor:0.0995543311505274 - perf/total_num_tokens:154302 - perf/time_per_step:30.159771636128426 - perf/throughput:639.5190995708728 (TaskRunner pid=2026458) Training Progress: 44%|████▍ | 44/100 [32:46<36:19, 38.91s/it] (TaskRunner pid=2026458) step:45 - global_seqlen/min:15729 - global_seqlen/max:24872 - global_seqlen/minmax_diff:9143 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18456.25 - actor/entropy:0.28802311420440674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29810237884521484 - training/rollout_probs_diff_mean:0.005760597065091133 - training/rollout_probs_diff_std:0.013836598955094814 - training/rollout_actor_probs_pearson_corr:0.9983675479888916 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.000093698501587 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.471605279832147e-05 - rollout_corr/rollout_is_max:1.7223904132843018 - rollout_corr/rollout_is_min:0.3712316155433655 - rollout_corr/rollout_is_std:0.04281449690461159 - rollout_corr/rollout_is_eff_sample_size:0.9981707478872216 - rollout_corr/rollout_is_seq_mean:1.000179648399353 - rollout_corr/rollout_is_seq_std:0.002977126045152545 - rollout_corr/rollout_is_seq_max:1.008954644203186 - rollout_corr/rollout_is_seq_min:0.9924071431159973 - rollout_corr/rollout_is_seq_max_deviation:0.008954644203186035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3517238958738744) - rollout_corr/training_log_ppl:np.float64(0.2982309421640821) - rollout_corr/kl:np.float64(0.0008551144521504739) - rollout_corr/k3_kl:np.float64(0.0011088182236562716) - rollout_corr/rollout_ppl:np.float64(1.350550597999245) - rollout_corr/rollout_log_ppl:np.float64(0.29737582663074136) - rollout_corr/log_ppl_diff:np.float64(0.0008551155333407223) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025770837091840804) - rollout_corr/log_ppl_diff_max:np.float64(0.009607642889022827) - rollout_corr/log_ppl_diff_min:np.float64(-0.007143139839172363) - rollout_corr/ppl_ratio:np.float64(1.000860691536218) - rollout_corr/chi2_token:np.float64(0.002733499277383089) - rollout_corr/chi2_seq:np.float64(0.8130777420010418) - actor/pg_loss:np.float64(0.002860508277080953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007193945029939641) - actor/ppo_kl:np.float64(8.01470097968604e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(108369.78586033895) - self_distillation/token_reweight_w_std:np.float64(1441947.6739233742) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984894776716828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07410955597879365) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.417113721370697) - perf/mfu/actor:np.float64(0.03440527561058198) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.48589324951172) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00041734983096830547 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00041734983096830547 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:262.0703125 - response_length/max:648.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.0703125 - response_length_non_aborted/max:648.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.201839566230774e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3249220605939627) - timing_s/agent_loop/generate_sequences/max:np.float64(5.273081086575985) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.846716286490846) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.273081086575985) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:418 - timing_s/agent_loop/slowest/response_length:648 - timing_s/gen:11.049067402258515 - timing_s/reward:0.0605863519012928 - timing_s/old_log_prob:2.241894433274865 - timing_s/adv:0.5028043333441019 - timing_s/update_actor:15.405420746654272 - timing_s/step:29.348894773051143 - timing_s/stop_profile:0.00011529400944709778 - timing_per_token_ms/adv:0.0034053798397839616 - timing_per_token_ms/gen:0.16469022808553457 - timing_per_token_ms/update_actor:0.1043374246302355 - perf/total_num_tokens:147650 - perf/time_per_step:29.348894773051143 - perf/throughput:628.8567301330532 (TaskRunner pid=2026458) Training Progress: 45%|████▌ | 45/100 [33:15<33:03, 36.06s/it] (TaskRunner pid=2026458) step:46 - global_seqlen/min:16485 - global_seqlen/max:20444 - global_seqlen/minmax_diff:3959 - global_seqlen/balanced_min:17819 - global_seqlen/balanced_max:17823 - global_seqlen/mean:17821.125 - actor/entropy:0.29167261719703674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032017827033997 - training/rollout_probs_diff_mean:0.005424427799880505 - training/rollout_probs_diff_std:0.013135196641087532 - training/rollout_actor_probs_pearson_corr:0.9985377788543701 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998041391372681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7731173759093508e-05 - rollout_corr/rollout_is_max:1.7920777797698975 - rollout_corr/rollout_is_min:0.19150957465171814 - rollout_corr/rollout_is_std:0.03981738165020943 - rollout_corr/rollout_is_eff_sample_size:0.9984166697314456 - rollout_corr/rollout_is_seq_mean:0.9997422695159912 - rollout_corr/rollout_is_seq_std:0.0028411736711859703 - rollout_corr/rollout_is_seq_max:1.0074349641799927 - rollout_corr/rollout_is_seq_min:0.9918615221977234 - rollout_corr/rollout_is_seq_max_deviation:0.008138477802276611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3577020564116538) - rollout_corr/training_log_ppl:np.float64(0.3013367626699619) - rollout_corr/kl:np.float64(0.0011281410026526828) - rollout_corr/k3_kl:np.float64(0.000916888057986398) - rollout_corr/rollout_ppl:np.float64(1.3561205281876028) - rollout_corr/rollout_log_ppl:np.float64(0.3002086222113576) - rollout_corr/log_ppl_diff:np.float64(0.0011281404586043209) - rollout_corr/log_ppl_abs_diff:np.float64(0.00256365115637891) - rollout_corr/log_ppl_diff_max:np.float64(0.008949041366577148) - rollout_corr/log_ppl_diff_min:np.float64(-0.009114667773246765) - rollout_corr/ppl_ratio:np.float64(1.0011336989700794) - rollout_corr/chi2_token:np.float64(0.0014092018827795982) - rollout_corr/chi2_seq:np.float64(0.8297181797679514) - actor/pg_loss:np.float64(0.005181095330044627) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006262393026190693) - actor/ppo_kl:np.float64(4.9855951789634645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(97563.27732459153) - self_distillation/token_reweight_w_std:np.float64(1470502.0369112268) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995444938540459) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10749436303740367) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.4946809858083725) - perf/mfu/actor:np.float64(0.03312182529882031) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.53104400634766) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002026108792051673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002026108792051673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.72265625 - response_length/max:593.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.72265625 - response_length_non_aborted/max:593.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.71662050485611e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3297640141099691) - timing_s/agent_loop/generate_sequences/max:np.float64(5.09482192248106) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.099804258738004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.09482192248106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:593 - timing_s/gen:11.312629146501422 - timing_s/reward:0.05594059266149998 - timing_s/old_log_prob:2.290945826098323 - timing_s/adv:0.4917367957532406 - timing_s/update_actor:15.34849851578474 - timing_s/step:29.586808905005455 - timing_s/stop_profile:0.00011978484690189362 - timing_per_token_ms/adv:0.0034491144340862362 - timing_per_token_ms/gen:0.15685624362531608 - timing_per_token_ms/update_actor:0.10765663303933352 - perf/total_num_tokens:142569 - perf/time_per_step:29.586808905005455 - perf/throughput:602.3334607398315 (TaskRunner pid=2026458) Training Progress: 46%|████▌ | 46/100 [33:45<30:43, 34.13s/it] (TaskRunner pid=2026458) step:47 - global_seqlen/min:14094 - global_seqlen/max:21506 - global_seqlen/minmax_diff:7412 - global_seqlen/balanced_min:18641 - global_seqlen/balanced_max:18644 - global_seqlen/mean:18642.625 - actor/entropy:0.27096572518348694 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7320154905319214 - training/rollout_probs_diff_mean:0.005300942808389664 - training/rollout_probs_diff_std:0.015477053821086884 - training/rollout_actor_probs_pearson_corr:0.9979209303855896 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0002624988555908 - rollout_corr/rollout_is_ratio_fraction_high:1.350238289887784e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017553097859490663 - rollout_corr/rollout_is_max:5.250345230102539 - rollout_corr/rollout_is_min:0.1398765593767166 - rollout_corr/rollout_is_std:0.041101276874542236 - rollout_corr/rollout_is_eff_sample_size:0.9983143657831011 - rollout_corr/rollout_is_seq_mean:1.0002628564834595 - rollout_corr/rollout_is_seq_std:0.0028906052466481924 - rollout_corr/rollout_is_seq_max:1.0110986232757568 - rollout_corr/rollout_is_seq_min:0.9913266897201538 - rollout_corr/rollout_is_seq_max_deviation:0.011098623275756836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.33038782607764) - rollout_corr/training_log_ppl:np.float64(0.2808810631977394) - rollout_corr/kl:np.float64(0.0008357813613741882) - rollout_corr/k3_kl:np.float64(0.0010391420327664491) - rollout_corr/rollout_ppl:np.float64(1.329236305784434) - rollout_corr/rollout_log_ppl:np.float64(0.28004528186284006) - rollout_corr/log_ppl_diff:np.float64(0.0008357813348993659) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023171580396592617) - rollout_corr/log_ppl_diff_max:np.float64(0.011047214269638062) - rollout_corr/log_ppl_diff_min:np.float64(-0.006683439016342163) - rollout_corr/ppl_ratio:np.float64(1.0008402362000197) - rollout_corr/chi2_token:np.float64(0.002344296546652913) - rollout_corr/chi2_seq:np.float64(0.7856939071789384) - actor/pg_loss:np.float64(0.0035682206507772207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00080771961438586) - actor/ppo_kl:np.float64(0.000137977912149978) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(180196.61803952185) - self_distillation/token_reweight_w_std:np.float64(1850028.0704001258) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998111962340772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09769563036388718) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4677669107913971) - perf/mfu/actor:np.float64(0.03410809941893227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.31404495239258) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003762101521715522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003762101521715522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.30078125 - response_length/max:738.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.30078125 - response_length_non_aborted/max:738.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.999237954616547e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5701810363680124) - timing_s/agent_loop/generate_sequences/max:np.float64(5.741097932681441) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.086432929841976) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.741097932681441) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:738 - timing_s/gen:12.44257121346891 - timing_s/reward:0.11020655557513237 - timing_s/old_log_prob:3.6139515712857246 - timing_s/adv:0.5418413802981377 - timing_s/update_actor:15.522931521758437 - timing_s/step:32.423666927963495 - timing_s/stop_profile:0.00011558458209037781 - timing_per_token_ms/adv:0.0036330813143142238 - timing_per_token_ms/gen:0.16800436415210313 - timing_per_token_ms/update_actor:0.10408225452262247 - perf/total_num_tokens:149141 - perf/time_per_step:32.423666927963495 - perf/throughput:574.9696677250851 (TaskRunner pid=2026458) Training Progress: 47%|████▋ | 47/100 [34:17<29:42, 33.63s/it] (TaskRunner pid=2026458) step:48 - global_seqlen/min:14172 - global_seqlen/max:18922 - global_seqlen/minmax_diff:4750 - global_seqlen/balanced_min:17273 - global_seqlen/balanced_max:17276 - global_seqlen/mean:17275.125 - actor/entropy:0.28335925936698914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2923179566860199 - training/rollout_probs_diff_mean:0.005517815239727497 - training/rollout_probs_diff_std:0.0134055744856596 - training/rollout_actor_probs_pearson_corr:0.9984236359596252 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999392032623291 - rollout_corr/rollout_is_ratio_fraction_high:1.4249682863010094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8499365726020187e-05 - rollout_corr/rollout_is_max:2.0768961906433105 - rollout_corr/rollout_is_min:0.30299994349479675 - rollout_corr/rollout_is_std:0.04144066944718361 - rollout_corr/rollout_is_eff_sample_size:0.9982854963367315 - rollout_corr/rollout_is_seq_mean:0.9998867511749268 - rollout_corr/rollout_is_seq_std:0.002759289462119341 - rollout_corr/rollout_is_seq_max:1.0119271278381348 - rollout_corr/rollout_is_seq_min:0.9927634000778198 - rollout_corr/rollout_is_seq_max_deviation:0.011927127838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3371913330629468) - rollout_corr/training_log_ppl:np.float64(0.28653379750903696) - rollout_corr/kl:np.float64(0.0009971100477166317) - rollout_corr/k3_kl:np.float64(0.0009379864269476457) - rollout_corr/rollout_ppl:np.float64(1.3358645238913596) - rollout_corr/rollout_log_ppl:np.float64(0.28553668738459237) - rollout_corr/log_ppl_diff:np.float64(0.000997110124444589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023510434839408845) - rollout_corr/log_ppl_diff_max:np.float64(0.009961530566215515) - rollout_corr/log_ppl_diff_min:np.float64(-0.00725024938583374) - rollout_corr/ppl_ratio:np.float64(1.0010016579180956) - rollout_corr/chi2_token:np.float64(0.0017369601409882307) - rollout_corr/chi2_seq:np.float64(0.5906375367194414) - actor/pg_loss:np.float64(0.0036190334940329194) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005808295049973822) - actor/ppo_kl:np.float64(-3.3437734998642554e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(104527.51239224058) - self_distillation/token_reweight_w_std:np.float64(1541202.450584725) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000157124362886) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09064709232188761) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.42223525047302246) - perf/mfu/actor:np.float64(0.031950463769025786) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.0163230895996) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019717998802661896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019717998802661896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.12890625 - response_length/max:573.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.12890625 - response_length_non_aborted/max:573.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012047216296195984 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2815109174698591) - timing_s/agent_loop/generate_sequences/max:np.float64(4.910025550052524) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0799834246499813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.910025550052524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:573 - timing_s/gen:10.97082038037479 - timing_s/reward:0.05485686659812927 - timing_s/old_log_prob:2.286929327994585 - timing_s/adv:0.5000464376062155 - timing_s/update_actor:15.382348308339715 - timing_s/step:29.284351360052824 - timing_s/stop_profile:0.0001140497624874115 - timing_per_token_ms/adv:0.0036182548433529096 - timing_per_token_ms/gen:0.15633071206199736 - timing_per_token_ms/update_actor:0.11130417513867277 - perf/total_num_tokens:138201 - perf/time_per_step:29.284351360052824 - perf/throughput:589.9097708397677 (TaskRunner pid=2026458) Training Progress: 48%|████▊ | 48/100 [34:46<28:01, 32.34s/it] (TaskRunner pid=2026458) step:49 - global_seqlen/min:17224 - global_seqlen/max:19716 - global_seqlen/minmax_diff:2492 - global_seqlen/balanced_min:18546 - global_seqlen/balanced_max:18550 - global_seqlen/mean:18548.75 - actor/entropy:0.27977773547172546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583575189113617 - training/rollout_probs_diff_mean:0.005447280127555132 - training/rollout_probs_diff_std:0.013272756710648537 - training/rollout_actor_probs_pearson_corr:0.9984596967697144 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998903274536133 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.59693042607978e-05 - rollout_corr/rollout_is_max:1.740144968032837 - rollout_corr/rollout_is_min:0.2820626199245453 - rollout_corr/rollout_is_std:0.04080290347337723 - rollout_corr/rollout_is_eff_sample_size:0.9983375338303951 - rollout_corr/rollout_is_seq_mean:0.9998434782028198 - rollout_corr/rollout_is_seq_std:0.0024030529893934727 - rollout_corr/rollout_is_seq_max:1.0073314905166626 - rollout_corr/rollout_is_seq_min:0.9888448715209961 - rollout_corr/rollout_is_seq_max_deviation:0.011155128479003906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3396195233799517) - rollout_corr/training_log_ppl:np.float64(0.28772373546962626) - rollout_corr/kl:np.float64(0.0007274425354566461) - rollout_corr/k3_kl:np.float64(0.0009657581270516857) - rollout_corr/rollout_ppl:np.float64(1.338642387650907) - rollout_corr/rollout_log_ppl:np.float64(0.2869962909608148) - rollout_corr/log_ppl_diff:np.float64(0.0007274445088114589) - rollout_corr/log_ppl_abs_diff:np.float64(0.002191261184634641) - rollout_corr/log_ppl_diff_max:np.float64(0.011016637086868286) - rollout_corr/log_ppl_diff_min:np.float64(-0.009674102067947388) - rollout_corr/ppl_ratio:np.float64(1.0007315212860703) - rollout_corr/chi2_token:np.float64(0.002396710915490985) - rollout_corr/chi2_seq:np.float64(1.0418477742932737) - actor/pg_loss:np.float64(0.003264832077547908) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005003044952900382) - actor/ppo_kl:np.float64(-0.0002956932246789279) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(133482.32340499433) - self_distillation/token_reweight_w_std:np.float64(1837865.312099801) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993252530694008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07909147051395848) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.44828861951828003) - perf/mfu/actor:np.float64(0.03315053423820138) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(151.27535438537598) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005617484916001558 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005617484916001558 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.8359375 - response_length/max:722.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.8359375 - response_length_non_aborted/max:722.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001618172973394394 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2382317576557398) - timing_s/agent_loop/generate_sequences/max:np.float64(5.747685985639691) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2742665582336485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.747685985639691) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:358 - timing_s/agent_loop/slowest/response_length:722 - timing_s/gen:11.949256183579564 - timing_s/reward:0.0566436555236578 - timing_s/old_log_prob:2.331390691921115 - timing_s/adv:0.5100956819951534 - timing_s/update_actor:16.06329415179789 - timing_s/step:30.9992211535573 - timing_s/stop_profile:0.00011705979704856873 - timing_per_token_ms/adv:0.003437534079083182 - timing_per_token_ms/gen:0.15515693488949495 - timing_per_token_ms/update_actor:0.1082505165563575 - perf/total_num_tokens:148390 - perf/time_per_step:30.9992211535573 - perf/throughput:598.3618074827485 (TaskRunner pid=2026458) Training Progress: 49%|████▉ | 49/100 [35:17<27:09, 31.95s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options matches the structure of the reactant that would form this product through a chemical reaction. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a structure with a substituted benzene ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). The correct reactant should include these key functional groups and match the overall structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a substituted benzene ring (c2ccccc2) and a nitro group (N), which is consistent with the product's structure. It also includes a chlorine atom and a trifluoromethyl group, making it a strong candidate for the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a chlorine atom and a substituted benzene ring, but it lacks the trifluoromethyl group and does not match the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (an oxygen atom) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (a double bond) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains the necessary functional groups and matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2026458) step:50 - global_seqlen/min:15714 - global_seqlen/max:24076 - global_seqlen/minmax_diff:8362 - global_seqlen/balanced_min:18055 - global_seqlen/balanced_max:18058 - global_seqlen/mean:18056.5 - actor/entropy:0.24377241730690002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36303967237472534 - training/rollout_probs_diff_mean:0.005148114170879126 - training/rollout_probs_diff_std:0.013693224638700485 - training/rollout_actor_probs_pearson_corr:0.9982551336288452 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:2.7458949261927046e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.864737224532291e-05 - rollout_corr/rollout_is_max:2.6678342819213867 - rollout_corr/rollout_is_min:0.17930372059345245 - rollout_corr/rollout_is_std:0.04035785421729088 - rollout_corr/rollout_is_eff_sample_size:0.9983741296547397 - rollout_corr/rollout_is_seq_mean:1.0000826120376587 - rollout_corr/rollout_is_seq_std:0.002955564297735691 - rollout_corr/rollout_is_seq_max:1.0125819444656372 - rollout_corr/rollout_is_seq_min:0.9909312725067139 - rollout_corr/rollout_is_seq_max_deviation:0.012581944465637207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3021241263486445) - rollout_corr/training_log_ppl:np.float64(0.26025468575244304) - rollout_corr/kl:np.float64(0.0007726539755310569) - rollout_corr/k3_kl:np.float64(0.0010959870580791176) - rollout_corr/rollout_ppl:np.float64(1.3010970023460686) - rollout_corr/rollout_log_ppl:np.float64(0.2594820303056622) - rollout_corr/log_ppl_diff:np.float64(0.0007726554467808455) - rollout_corr/log_ppl_abs_diff:np.float64(0.00248320066020824) - rollout_corr/log_ppl_diff_max:np.float64(0.010783776640892029) - rollout_corr/log_ppl_diff_min:np.float64(-0.014668971300125122) - rollout_corr/ppl_ratio:np.float64(1.0007780094165355) - rollout_corr/chi2_token:np.float64(0.002904584165662527) - rollout_corr/chi2_seq:np.float64(1.7037486750632524) - actor/pg_loss:np.float64(0.00390949088614434) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012861561140198319) - actor/ppo_kl:np.float64(-2.874969411337247e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(303182.6307455059) - self_distillation/token_reweight_w_std:np.float64(3853282.430746559) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001452625496313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08799264291883446) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.9146713092923164) - perf/mfu/actor:np.float64(0.03308314844340515) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.47300720214844) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/score/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06458122232163772) - val-core/sciknoweval/acc/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08550843930681744) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8024761904761903) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05941417072300563) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6757285714285716) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00038099291850812733 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00038099291850812733 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:284.515625 - response_length/max:754.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.515625 - response_length_non_aborted/max:754.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001567751169204712 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4153082128614187) - timing_s/agent_loop/generate_sequences/max:np.float64(6.037689724937081) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1098132879196783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.037689724937081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:754 - timing_s/gen:12.157338604331017 - timing_s/reward:0.07574324123561382 - timing_s/old_log_prob:2.3888504561036825 - timing_s/adv:0.52120690792799 - timing_s/update_actor:15.507932955399156 - timing_s/step:30.738427244126797 - timing_s/testing:64.3304407633841 - timing_s/save_checkpoint:6.778321286663413 - timing_s/stop_profile:0.00014533661305904388 - timing_per_token_ms/adv:0.0036081667815467417 - timing_per_token_ms/gen:0.16691386957453755 - timing_per_token_ms/update_actor:0.10735699717137288 - perf/total_num_tokens:144452 - perf/time_per_step:30.738427244126797 - perf/throughput:587.4243290521658 (TaskRunner pid=2026458) Training Progress: 50%|█████ | 50/100 [36:59<44:06, 52.94s/it] (TaskRunner pid=2026458) step:51 - global_seqlen/min:17642 - global_seqlen/max:21320 - global_seqlen/minmax_diff:3678 - global_seqlen/balanced_min:19266 - global_seqlen/balanced_max:19274 - global_seqlen/mean:19272.125 - actor/entropy:0.27418410778045654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3186328411102295 - training/rollout_probs_diff_mean:0.005429690238088369 - training/rollout_probs_diff_std:0.013169948011636734 - training/rollout_actor_probs_pearson_corr:0.9984568953514099 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999269247055054 - rollout_corr/rollout_is_ratio_fraction_high:2.6965442884829827e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6965442884829827e-05 - rollout_corr/rollout_is_max:2.9393632411956787 - rollout_corr/rollout_is_min:0.18107180297374725 - rollout_corr/rollout_is_std:0.04137301445007324 - rollout_corr/rollout_is_eff_sample_size:0.9982909612066878 - rollout_corr/rollout_is_seq_mean:0.9999714493751526 - rollout_corr/rollout_is_seq_std:0.0026941560208797455 - rollout_corr/rollout_is_seq_max:1.0119987726211548 - rollout_corr/rollout_is_seq_min:0.9906357526779175 - rollout_corr/rollout_is_seq_max_deviation:0.011998772621154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3326939661055803) - rollout_corr/training_log_ppl:np.float64(0.283328754856484) - rollout_corr/kl:np.float64(0.001354399174752885) - rollout_corr/k3_kl:np.float64(0.0011438059663078093) - rollout_corr/rollout_ppl:np.float64(1.3308947938494384) - rollout_corr/rollout_log_ppl:np.float64(0.2819743549771374) - rollout_corr/log_ppl_diff:np.float64(0.0013543998793466017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025673647614894435) - rollout_corr/log_ppl_diff_max:np.float64(0.017022624611854553) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628902792930603) - rollout_corr/ppl_ratio:np.float64(1.001360428519547) - rollout_corr/chi2_token:np.float64(0.0018329326994717121) - rollout_corr/chi2_seq:np.float64(1.2063880120404065) - actor/pg_loss:np.float64(0.006517422734759748) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015835570143281075) - actor/ppo_kl:np.float64(0.0004230728660630234) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(238186.0888677421) - self_distillation/token_reweight_w_std:np.float64(3422829.743828913) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022471062839031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1062716192973312) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.5796995311975479) - perf/mfu/actor:np.float64(0.035632098241036) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.68563079833984) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010413717478513718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010413717478513718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.72265625 - response_length/max:773.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.72265625 - response_length_non_aborted/max:773.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014895200729370117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6900891289114952) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7452072985470295) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1825576704068226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7452072985470295) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:773 - timing_s/gen:11.57777032442391 - timing_s/reward:0.05630043335258961 - timing_s/old_log_prob:2.3727036882191896 - timing_s/adv:0.5643414072692394 - timing_s/update_actor:15.28370295651257 - timing_s/step:29.95019925571978 - timing_s/stop_profile:0.00013649649918079376 - timing_per_token_ms/adv:0.003660347569801199 - timing_per_token_ms/gen:0.15609985741244872 - timing_per_token_ms/update_actor:0.09913088824216693 - perf/total_num_tokens:154177 - perf/time_per_step:29.95019925571978 - perf/throughput:643.472346726357 (TaskRunner pid=2026458) Training Progress: 51%|█████ | 51/100 [37:29<37:36, 46.06s/it] (TaskRunner pid=2026458) step:52 - global_seqlen/min:14188 - global_seqlen/max:21038 - global_seqlen/minmax_diff:6850 - global_seqlen/balanced_min:18334 - global_seqlen/balanced_max:18335 - global_seqlen/mean:18334.5 - actor/entropy:0.2573181688785553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730220556259155 - training/rollout_probs_diff_mean:0.005135765764862299 - training/rollout_probs_diff_std:0.012896839529275894 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9999938011169434 - rollout_corr/rollout_is_ratio_fraction_high:2.519780355214607e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.519780355214607e-05 - rollout_corr/rollout_is_max:2.1215898990631104 - rollout_corr/rollout_is_min:0.3974515199661255 - rollout_corr/rollout_is_std:0.040721017867326736 - rollout_corr/rollout_is_eff_sample_size:0.9983446626766718 - rollout_corr/rollout_is_seq_mean:0.9999977946281433 - rollout_corr/rollout_is_seq_std:0.0026824194937944412 - rollout_corr/rollout_is_seq_max:1.0067678689956665 - rollout_corr/rollout_is_seq_min:0.9906572103500366 - rollout_corr/rollout_is_seq_max_deviation:0.009342789649963379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3137918766587973) - rollout_corr/training_log_ppl:np.float64(0.2694710589275928) - rollout_corr/kl:np.float64(0.0010667287329408914) - rollout_corr/k3_kl:np.float64(0.0010078331736167456) - rollout_corr/rollout_ppl:np.float64(1.3123429850675166) - rollout_corr/rollout_log_ppl:np.float64(0.26840433149482124) - rollout_corr/log_ppl_diff:np.float64(0.0010667274327715859) - rollout_corr/log_ppl_abs_diff:np.float64(0.002380369245656766) - rollout_corr/log_ppl_diff_max:np.float64(0.010203629732131958) - rollout_corr/log_ppl_diff_min:np.float64(-0.006238669157028198) - rollout_corr/ppl_ratio:np.float64(1.0010714137461036) - rollout_corr/chi2_token:np.float64(0.001948290504515171) - rollout_corr/chi2_seq:np.float64(0.5412429641000926) - actor/pg_loss:np.float64(0.003885703510604799) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619293583549734) - actor/ppo_kl:np.float64(0.00019392038652732424) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(186537.93477050052) - self_distillation/token_reweight_w_std:np.float64(2011255.6988170468) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005899087991565) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06515720163588412) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4466311186552048) - perf/mfu/actor:np.float64(0.03402000847749656) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.35942840576172) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01099569071084261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01099569071084261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:310.046875 - response_length/max:700.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.046875 - response_length_non_aborted/max:700.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014230050146579742 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4891746453940868) - timing_s/agent_loop/generate_sequences/max:np.float64(5.560187539085746) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.207101171356044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.560187539085746) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:700 - timing_s/gen:11.716578045859933 - timing_s/reward:0.05753312073647976 - timing_s/old_log_prob:2.249606389552355 - timing_s/adv:0.5425826162099838 - timing_s/update_actor:15.384762125089765 - timing_s/step:30.04255287349224 - timing_s/stop_profile:0.00013405084609985352 - timing_per_token_ms/adv:0.003699191525607351 - timing_per_token_ms/gen:0.14761601126165314 - timing_per_token_ms/update_actor:0.10488943061639099 - perf/total_num_tokens:146676 - perf/time_per_step:30.04255287349224 - perf/throughput:610.2843549017191 (TaskRunner pid=2026458) Training Progress: 52%|█████▏ | 52/100 [37:59<33:00, 41.27s/it] (TaskRunner pid=2026458) step:53 - global_seqlen/min:17379 - global_seqlen/max:20313 - global_seqlen/minmax_diff:2934 - global_seqlen/balanced_min:18481 - global_seqlen/balanced_max:18482 - global_seqlen/mean:18481.75 - actor/entropy:0.278646856546402 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462908267974854 - training/rollout_probs_diff_mean:0.005563129670917988 - training/rollout_probs_diff_std:0.013384878635406494 - training/rollout_actor_probs_pearson_corr:0.9984221458435059 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999546408653259 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5409334891010076e-05 - rollout_corr/rollout_is_max:1.8313579559326172 - rollout_corr/rollout_is_min:0.0737028419971466 - rollout_corr/rollout_is_std:0.04174019396305084 - rollout_corr/rollout_is_eff_sample_size:0.9982605487774039 - rollout_corr/rollout_is_seq_mean:0.9999428391456604 - rollout_corr/rollout_is_seq_std:0.0028358809649944305 - rollout_corr/rollout_is_seq_max:1.0075714588165283 - rollout_corr/rollout_is_seq_min:0.9910898804664612 - rollout_corr/rollout_is_seq_max_deviation:0.008910119533538818 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338077249005437) - rollout_corr/training_log_ppl:np.float64(0.2873227519157808) - rollout_corr/kl:np.float64(0.0011784572657376202) - rollout_corr/k3_kl:np.float64(0.0013956419594478575) - rollout_corr/rollout_ppl:np.float64(1.3364087981171906) - rollout_corr/rollout_log_ppl:np.float64(0.2861442923604045) - rollout_corr/log_ppl_diff:np.float64(0.001178459555376321) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027842955314554274) - rollout_corr/log_ppl_diff_max:np.float64(0.0132656991481781) - rollout_corr/log_ppl_diff_min:np.float64(-0.010441526770591736) - rollout_corr/ppl_ratio:np.float64(1.0011851615272462) - rollout_corr/chi2_token:np.float64(0.003180387197062373) - rollout_corr/chi2_seq:np.float64(1.2029992926400155) - actor/pg_loss:np.float64(0.003771678893826902) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009931258632605022) - actor/ppo_kl:np.float64(0.00019185108615538127) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(52868.786943790736) - self_distillation/token_reweight_w_std:np.float64(637934.6799204362) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990235811565071) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07278861297527328) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.44415684044361115) - perf/mfu/actor:np.float64(0.0346355208538403) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.01051712036133) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006060396321117878 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006060396321117878 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:281.9921875 - response_length/max:616.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.9921875 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001376233994960785 - timing_s/agent_loop/generate_sequences/min:np.float64(1.366797188296914) - timing_s/agent_loop/generate_sequences/max:np.float64(5.209049364551902) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1154251298939926) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.209049364551902) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:227 - timing_s/agent_loop/slowest/response_length:616 - timing_s/gen:11.11843347735703 - timing_s/reward:0.05367938429117203 - timing_s/old_log_prob:2.252691153436899 - timing_s/adv:0.4951089806854725 - timing_s/update_actor:15.301375834271312 - timing_s/step:29.318861059844494 - timing_s/stop_profile:0.00011683069169521332 - timing_per_token_ms/adv:0.0033486343330953 - timing_per_token_ms/gen:0.15401625540042985 - timing_per_token_ms/update_actor:0.10348976581134979 - perf/total_num_tokens:147854 - perf/time_per_step:29.318861059844494 - perf/throughput:630.3706669326542 (TaskRunner pid=2026458) Training Progress: 53%|█████▎ | 53/100 [38:29<29:31, 37.70s/it] (TaskRunner pid=2026458) step:54 - global_seqlen/min:15111 - global_seqlen/max:18863 - global_seqlen/minmax_diff:3752 - global_seqlen/balanced_min:17356 - global_seqlen/balanced_max:17358 - global_seqlen/mean:17357.625 - actor/entropy:0.2270389199256897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413564920425415 - training/rollout_probs_diff_mean:0.005126654170453548 - training/rollout_probs_diff_std:0.01343945786356926 - training/rollout_actor_probs_pearson_corr:0.9981903433799744 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001782178878784 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.876001235563308e-05 - rollout_corr/rollout_is_max:1.8594493865966797 - rollout_corr/rollout_is_min:0.35453474521636963 - rollout_corr/rollout_is_std:0.04036819189786911 - rollout_corr/rollout_is_eff_sample_size:0.998373654367098 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002654197160154581 - rollout_corr/rollout_is_seq_max:1.0092252492904663 - rollout_corr/rollout_is_seq_min:0.9911713600158691 - rollout_corr/rollout_is_seq_max_deviation:0.009225249290466309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2745554209686816) - rollout_corr/training_log_ppl:np.float64(0.23950671087368391) - rollout_corr/kl:np.float64(0.0011859710301269644) - rollout_corr/k3_kl:np.float64(0.0012516252674004136) - rollout_corr/rollout_ppl:np.float64(1.2729601715691388) - rollout_corr/rollout_log_ppl:np.float64(0.23832074020174332) - rollout_corr/log_ppl_diff:np.float64(0.001185970671940595) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027243185904808342) - rollout_corr/log_ppl_diff_max:np.float64(0.01128050684928894) - rollout_corr/log_ppl_diff_min:np.float64(-0.011040359735488892) - rollout_corr/ppl_ratio:np.float64(1.0011925832368433) - rollout_corr/chi2_token:np.float64(0.0025079476181417704) - rollout_corr/chi2_seq:np.float64(1.487264727940783) - actor/pg_loss:np.float64(0.0043368348851799965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017240041906916304) - actor/ppo_kl:np.float64(0.0004469881809043841) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(275413.10884484975) - self_distillation/token_reweight_w_std:np.float64(3763164.204352789) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020550324115902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08829297052579932) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5274116396903992) - perf/mfu/actor:np.float64(0.03254539451835227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.84357070922852) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047957319766283035 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047957319766283035 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.64453125 - response_length/max:650.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.64453125 - response_length_non_aborted/max:650.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.452683687210083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3436731528490782) - timing_s/agent_loop/generate_sequences/max:np.float64(5.206967985257506) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.978192554124689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.206967985257506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:650 - timing_s/gen:10.970427636057138 - timing_s/reward:0.05443461053073406 - timing_s/old_log_prob:2.2816072031855583 - timing_s/adv:0.47508152201771736 - timing_s/update_actor:15.217902828007936 - timing_s/step:29.08791142515838 - timing_s/stop_profile:0.0001310836523771286 - timing_per_token_ms/adv:0.0034212739503367926 - timing_per_token_ms/gen:0.15775481566352423 - timing_per_token_ms/update_actor:0.10959090621562524 - perf/total_num_tokens:138861 - perf/time_per_step:29.08791142515838 - perf/throughput:596.7298492592096 (TaskRunner pid=2026458) Training Progress: 54%|█████▍ | 54/100 [38:58<26:55, 35.13s/it] (TaskRunner pid=2026458) step:55 - global_seqlen/min:15554 - global_seqlen/max:21932 - global_seqlen/minmax_diff:6378 - global_seqlen/balanced_min:18814 - global_seqlen/balanced_max:18817 - global_seqlen/mean:18816.625 - actor/entropy:0.22956356406211853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9232618808746338 - training/rollout_probs_diff_mean:0.005393934901803732 - training/rollout_probs_diff_std:0.014924020506441593 - training/rollout_actor_probs_pearson_corr:0.9978342056274414 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000108242034912 - rollout_corr/rollout_is_ratio_fraction_high:4.133313268539496e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.133313268539496e-05 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.04885798692703247 - rollout_corr/rollout_is_std:0.042956262826919556 - rollout_corr/rollout_is_eff_sample_size:0.9981586331310626 - rollout_corr/rollout_is_seq_mean:1.0001986026763916 - rollout_corr/rollout_is_seq_std:0.002816676627844572 - rollout_corr/rollout_is_seq_max:1.0112348794937134 - rollout_corr/rollout_is_seq_min:0.9917329549789429 - rollout_corr/rollout_is_seq_max_deviation:0.011234879493713379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2765053128823638) - rollout_corr/training_log_ppl:np.float64(0.24006979388650507) - rollout_corr/kl:np.float64(0.0010645344639215182) - rollout_corr/k3_kl:np.float64(0.0011318304883047858) - rollout_corr/rollout_ppl:np.float64(1.275088603142649) - rollout_corr/rollout_log_ppl:np.float64(0.23900525833596475) - rollout_corr/log_ppl_diff:np.float64(0.0010645355505403131) - rollout_corr/log_ppl_abs_diff:np.float64(0.002453350374707952) - rollout_corr/log_ppl_diff_max:np.float64(0.011057406663894653) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540911436080933) - rollout_corr/ppl_ratio:np.float64(1.0010695066303015) - rollout_corr/chi2_token:np.float64(0.0023642820306122303) - rollout_corr/chi2_seq:np.float64(0.9924367545172572) - actor/pg_loss:np.float64(0.005413581035099924) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001406622541253455) - actor/ppo_kl:np.float64(0.0002698558086962066) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(295807.0609528662) - self_distillation/token_reweight_w_std:np.float64(3781227.560846627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014416894409806) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09695331199327484) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5869030505418777) - perf/mfu/actor:np.float64(0.03432748799407685) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77687454223633) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014518261887133121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014518261887133121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:283.51953125 - response_length/max:687.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:283.51953125 - response_length_non_aborted/max:687.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010005757212638855 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9696572478860617) - timing_s/agent_loop/generate_sequences/max:np.float64(5.428103374317288) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.78926639315614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.428103374317288) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:687 - timing_s/gen:11.274114403873682 - timing_s/reward:0.0568949319422245 - timing_s/old_log_prob:2.307572791352868 - timing_s/adv:0.5059547368437052 - timing_s/update_actor:15.750575561076403 - timing_s/step:29.98292907886207 - timing_s/stop_profile:0.00012236833572387695 - timing_per_token_ms/adv:0.003361088511115205 - timing_per_token_ms/gen:0.1553314834994514 - timing_per_token_ms/update_actor:0.10463204454223594 - perf/total_num_tokens:150533 - perf/time_per_step:29.98292907886207 - perf/throughput:627.5779444532556 (TaskRunner pid=2026458) Training Progress: 55%|█████▌ | 55/100 [39:28<25:11, 33.60s/it] (TaskRunner pid=2026458) step:56 - global_seqlen/min:15183 - global_seqlen/max:19968 - global_seqlen/minmax_diff:4785 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16856 - global_seqlen/mean:16853.75 - actor/entropy:0.23840856552124023 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032567977905273 - training/rollout_probs_diff_mean:0.0051898774690926075 - training/rollout_probs_diff_std:0.013416245579719543 - training/rollout_actor_probs_pearson_corr:0.9982264637947083 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000948905944824 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9873039238736965e-05 - rollout_corr/rollout_is_max:1.868481159210205 - rollout_corr/rollout_is_min:0.42863455414772034 - rollout_corr/rollout_is_std:0.03991232439875603 - rollout_corr/rollout_is_eff_sample_size:0.9984097775176272 - rollout_corr/rollout_is_seq_mean:1.0001156330108643 - rollout_corr/rollout_is_seq_std:0.002804977586492896 - rollout_corr/rollout_is_seq_max:1.008453607559204 - rollout_corr/rollout_is_seq_min:0.9917566776275635 - rollout_corr/rollout_is_seq_max_deviation:0.008453607559204102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.288645532913506) - rollout_corr/training_log_ppl:np.float64(0.25009057341958396) - rollout_corr/kl:np.float64(0.001021610375019577) - rollout_corr/k3_kl:np.float64(0.0012247983307815957) - rollout_corr/rollout_ppl:np.float64(1.2872521225363016) - rollout_corr/rollout_log_ppl:np.float64(0.2490689625556115) - rollout_corr/log_ppl_diff:np.float64(0.0010216108639724553) - rollout_corr/log_ppl_abs_diff:np.float64(0.002677871671039611) - rollout_corr/log_ppl_diff_max:np.float64(0.011799484491348267) - rollout_corr/log_ppl_diff_min:np.float64(-0.013348877429962158) - rollout_corr/ppl_ratio:np.float64(1.001028069993481) - rollout_corr/chi2_token:np.float64(0.0027951118536293507) - rollout_corr/chi2_seq:np.float64(0.911775927990675) - actor/pg_loss:np.float64(0.0038659967249259353) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001200501193125092) - actor/ppo_kl:np.float64(0.0002928259359418206) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(173579.36585247447) - self_distillation/token_reweight_w_std:np.float64(2190372.348648581) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999903631862253) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08299089543288574) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5205514430999756) - perf/mfu/actor:np.float64(0.03136809590233023) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65859985351562) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015309933223761618 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015309933223761618 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.5234375 - response_length/max:725.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.5234375 - response_length_non_aborted/max:725.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001024492084980011 - timing_s/agent_loop/generate_sequences/min:np.float64(1.564612640067935) - timing_s/agent_loop/generate_sequences/max:np.float64(5.375423099845648) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8240807638067054) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.375423099845648) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:725 - timing_s/gen:11.348324349150062 - timing_s/reward:0.054198144003748894 - timing_s/old_log_prob:2.395893217995763 - timing_s/adv:0.4817402958869934 - timing_s/update_actor:15.269863871857524 - timing_s/step:29.636497838422656 - timing_s/stop_profile:0.00012066960334777832 - timing_per_token_ms/adv:0.003572945901409133 - timing_per_token_ms/gen:0.1695044712345043 - timing_per_token_ms/update_actor:0.11325271728738058 - perf/total_num_tokens:134830 - perf/time_per_step:29.636497838422656 - perf/throughput:568.6822407926255 (TaskRunner pid=2026458) Training Progress: 56%|█████▌ | 56/100 [39:58<23:46, 32.42s/it] (TaskRunner pid=2026458) step:57 - global_seqlen/min:15145 - global_seqlen/max:21004 - global_seqlen/minmax_diff:5859 - global_seqlen/balanced_min:17327 - global_seqlen/balanced_max:17330 - global_seqlen/mean:17329.125 - actor/entropy:0.2190825492143631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744122624397278 - training/rollout_probs_diff_mean:0.005394987761974335 - training/rollout_probs_diff_std:0.014672977849841118 - training/rollout_actor_probs_pearson_corr:0.9977869987487793 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0002177953720093 - rollout_corr/rollout_is_ratio_fraction_high:9.200055501423776e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013800083252135664 - rollout_corr/rollout_is_max:3.8171095848083496 - rollout_corr/rollout_is_min:0.15978100895881653 - rollout_corr/rollout_is_std:0.04392360523343086 - rollout_corr/rollout_is_eff_sample_size:0.9980752630793054 - rollout_corr/rollout_is_seq_mean:1.0002235174179077 - rollout_corr/rollout_is_seq_std:0.0027234358713030815 - rollout_corr/rollout_is_seq_max:1.0101075172424316 - rollout_corr/rollout_is_seq_min:0.9909296035766602 - rollout_corr/rollout_is_seq_max_deviation:0.01010751724243164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2523039323277771) - rollout_corr/training_log_ppl:np.float64(0.22247063313261606) - rollout_corr/kl:np.float64(0.0011533432969415003) - rollout_corr/k3_kl:np.float64(0.0011055539546305226) - rollout_corr/rollout_ppl:np.float64(1.2508012815378606) - rollout_corr/rollout_log_ppl:np.float64(0.22131728671956807) - rollout_corr/log_ppl_diff:np.float64(0.0011533464130479842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025634898629505187) - rollout_corr/log_ppl_diff_max:np.float64(0.016222700476646423) - rollout_corr/log_ppl_diff_min:np.float64(-0.007622018456459045) - rollout_corr/ppl_ratio:np.float64(1.0011591671500355) - rollout_corr/chi2_token:np.float64(0.0020452477037906647) - rollout_corr/chi2_seq:np.float64(0.5157537555787712) - actor/pg_loss:np.float64(0.003759522340260446) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008975071282293356) - actor/ppo_kl:np.float64(0.00035579913428307464) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(236077.81382388272) - self_distillation/token_reweight_w_std:np.float64(2308889.815033672) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994992383290082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0630278438620735) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.43273352086544037) - perf/mfu/actor:np.float64(0.031510354454649704) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.82762956619263) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027676832396537066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027676832396537066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.75390625 - response_length/max:747.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.75390625 - response_length_non_aborted/max:747.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014020688831806183 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4702318627387285) - timing_s/agent_loop/generate_sequences/max:np.float64(5.407115560024977) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.896707163978135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.407115560024977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:747 - timing_s/gen:11.385951856151223 - timing_s/reward:0.05382969416677952 - timing_s/old_log_prob:2.539514096453786 - timing_s/adv:0.49977708980441093 - timing_s/update_actor:15.644645780324936 - timing_s/step:30.209736336022615 - timing_s/stop_profile:0.00012044794857501984 - timing_per_token_ms/adv:0.0036050369666992053 - timing_per_token_ms/gen:0.17458564264150794 - timing_per_token_ms/update_actor:0.1128493632852563 - perf/total_num_tokens:138633 - perf/time_per_step:30.209736336022615 - perf/throughput:573.6271514338392 (TaskRunner pid=2026458) Training Progress: 57%|█████▋ | 57/100 [40:28<22:46, 31.77s/it] (TaskRunner pid=2026458) step:58 - global_seqlen/min:14734 - global_seqlen/max:19599 - global_seqlen/minmax_diff:4865 - global_seqlen/balanced_min:17102 - global_seqlen/balanced_max:17103 - global_seqlen/mean:17102.75 - actor/entropy:0.23608146607875824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5141049027442932 - training/rollout_probs_diff_mean:0.005427653901278973 - training/rollout_probs_diff_std:0.013945525512099266 - training/rollout_actor_probs_pearson_corr:0.9981228709220886 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999080896377563 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5163001990003977e-05 - rollout_corr/rollout_is_max:1.724382996559143 - rollout_corr/rollout_is_min:0.16556662321090698 - rollout_corr/rollout_is_std:0.04180582985281944 - rollout_corr/rollout_is_eff_sample_size:0.9982550842404107 - rollout_corr/rollout_is_seq_mean:0.9999476671218872 - rollout_corr/rollout_is_seq_std:0.002901083789765835 - rollout_corr/rollout_is_seq_max:1.00826895236969 - rollout_corr/rollout_is_seq_min:0.9917892217636108 - rollout_corr/rollout_is_seq_max_deviation:0.008268952369689941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2862667385488749) - rollout_corr/training_log_ppl:np.float64(0.24756408171379007) - rollout_corr/kl:np.float64(0.0008958796237372724) - rollout_corr/k3_kl:np.float64(0.0010313347952433105) - rollout_corr/rollout_ppl:np.float64(1.2851230534724891) - rollout_corr/rollout_log_ppl:np.float64(0.24666820294805802) - rollout_corr/log_ppl_diff:np.float64(0.0008958787657320499) - rollout_corr/log_ppl_abs_diff:np.float64(0.002455878769978881) - rollout_corr/log_ppl_diff_max:np.float64(0.014661163091659546) - rollout_corr/log_ppl_diff_min:np.float64(-0.006324887275695801) - rollout_corr/ppl_ratio:np.float64(1.00090107973665) - rollout_corr/chi2_token:np.float64(0.0023643041495233774) - rollout_corr/chi2_seq:np.float64(0.6059110292699188) - actor/pg_loss:np.float64(0.003710375865921378) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011193532754987245) - actor/ppo_kl:np.float64(-8.923532409355062e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(286496.5454768897) - self_distillation/token_reweight_w_std:np.float64(3628499.957205932) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005135682877153) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07967197702964768) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5009380057454109) - perf/mfu/actor:np.float64(0.03152191694270064) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.63263320922852) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016660349210724235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016660349210724235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.6171875 - response_length/max:583.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.6171875 - response_length_non_aborted/max:583.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001420155167579651 - timing_s/agent_loop/generate_sequences/min:np.float64(1.489874416962266) - timing_s/agent_loop/generate_sequences/max:np.float64(4.795747706666589) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8081408347570687) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.795747706666589) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:10.590774795040488 - timing_s/reward:0.05500875972211361 - timing_s/old_log_prob:2.199430391192436 - timing_s/adv:0.4823594633489847 - timing_s/update_actor:15.483584174886346 - timing_s/step:28.898630160838366 - timing_s/stop_profile:0.0001272875815629959 - timing_per_token_ms/adv:0.0035254525101883084 - timing_per_token_ms/gen:0.16058794230539028 - timing_per_token_ms/update_actor:0.11316589565191523 - perf/total_num_tokens:136822 - perf/time_per_step:28.898630160838366 - perf/throughput:591.818709219532 (TaskRunner pid=2026458) Training Progress: 58%|█████▊ | 58/100 [40:57<21:38, 30.92s/it] (TaskRunner pid=2026458) step:59 - global_seqlen/min:15244 - global_seqlen/max:19485 - global_seqlen/minmax_diff:4241 - global_seqlen/balanced_min:16769 - global_seqlen/balanced_max:16771 - global_seqlen/mean:16770.625 - actor/entropy:0.2315026819705963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5046374797821045 - training/rollout_probs_diff_mean:0.0052477153949439526 - training/rollout_probs_diff_std:0.013823322020471096 - training/rollout_actor_probs_pearson_corr:0.9981176853179932 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:1.0000110864639282 - rollout_corr/rollout_is_ratio_fraction_high:1.5287481801351532e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.114992720540613e-05 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.3053041398525238 - rollout_corr/rollout_is_std:0.04093707725405693 - rollout_corr/rollout_is_eff_sample_size:0.998326959562596 - rollout_corr/rollout_is_seq_mean:1.000074863433838 - rollout_corr/rollout_is_seq_std:0.0029152550268918276 - rollout_corr/rollout_is_seq_max:1.0087932348251343 - rollout_corr/rollout_is_seq_min:0.9920890927314758 - rollout_corr/rollout_is_seq_max_deviation:0.008793234825134277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2814294979907572) - rollout_corr/training_log_ppl:np.float64(0.24554033184540458) - rollout_corr/kl:np.float64(0.0010128226648049576) - rollout_corr/k3_kl:np.float64(0.0010760613429283694) - rollout_corr/rollout_ppl:np.float64(1.280145457945764) - rollout_corr/rollout_log_ppl:np.float64(0.2445275079808198) - rollout_corr/log_ppl_diff:np.float64(0.0010128238645847887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026435373874846846) - rollout_corr/log_ppl_diff_max:np.float64(0.01640312373638153) - rollout_corr/log_ppl_diff_min:np.float64(-0.01078614592552185) - rollout_corr/ppl_ratio:np.float64(1.001019265735522) - rollout_corr/chi2_token:np.float64(0.0022960123606026173) - rollout_corr/chi2_seq:np.float64(0.9392846249975264) - actor/pg_loss:np.float64(0.002857705345377326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044364196128299227) - actor/ppo_kl:np.float64(0.0001824798405234418) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(71359.55967221712) - self_distillation/token_reweight_w_std:np.float64(989988.4983619121) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995234010275453) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04141386490664445) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3792598769068718) - perf/mfu/actor:np.float64(0.03147454843432276) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65855026245117) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003848623251542449 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:0.003848623251542449 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:255.51953125 - response_length/max:621.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.51953125 - response_length_non_aborted/max:621.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.882924258708954e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.939049456268549) - timing_s/agent_loop/generate_sequences/max:np.float64(5.183893235400319) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.658139636150736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.183893235400319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:217 - timing_s/agent_loop/slowest/response_length:621 - timing_s/gen:10.904466819018126 - timing_s/reward:0.052795736119151115 - timing_s/old_log_prob:2.23296419903636 - timing_s/adv:0.47304221242666245 - timing_s/update_actor:15.1854842081666 - timing_s/step:28.936049316078424 - timing_s/stop_profile:5.451589822769165e-05 - timing_per_token_ms/adv:0.0035258242643510787 - timing_per_token_ms/gen:0.16670183020222473 - timing_per_token_ms/update_actor:0.11318513925514552 - perf/total_num_tokens:134165 - perf/time_per_step:28.936049316078424 - perf/throughput:579.575491346752 (TaskRunner pid=2026458) Training Progress: 59%|█████▉ | 59/100 [41:26<20:43, 30.33s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options contains the correct functional groups and structure that match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrogen-containing group (n1) connected to a cyclohexane ring (Cc2ccc(Cl)cc2), with a Cl group and a trifluoromethyl group (C(F)(F)F). The correct reactant must include these key features. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1) connected to a cyclohexane ring with a Cl group and a trifluoromethyl group. This matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it does not include the Cl group or the trifluoromethyl group in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (Oc2ccc(Cl)c(C(F)(F)F)c2) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (CC=C(Cc1ccccc1)C(F)(F)F) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2026458) step:60 - global_seqlen/min:14552 - global_seqlen/max:17730 - global_seqlen/minmax_diff:3178 - global_seqlen/balanced_min:16889 - global_seqlen/balanced_max:16891 - global_seqlen/mean:16890.25 - actor/entropy:0.258317232131958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851317644119263 - training/rollout_probs_diff_mean:0.005422248970717192 - training/rollout_probs_diff_std:0.013785075396299362 - training/rollout_actor_probs_pearson_corr:0.9982817769050598 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0001024007797241 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4916022337274626e-05 - rollout_corr/rollout_is_max:1.976409912109375 - rollout_corr/rollout_is_min:0.3921801745891571 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982955945128663 - rollout_corr/rollout_is_seq_mean:1.000079870223999 - rollout_corr/rollout_is_seq_std:0.002677295822650194 - rollout_corr/rollout_is_seq_max:1.0086641311645508 - rollout_corr/rollout_is_seq_min:0.9929971694946289 - rollout_corr/rollout_is_seq_max_deviation:0.008664131164550781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3128179563209414) - rollout_corr/training_log_ppl:np.float64(0.2678291550837457) - rollout_corr/kl:np.float64(0.001152293861238718) - rollout_corr/k3_kl:np.float64(0.0010784027733734547) - rollout_corr/rollout_ppl:np.float64(1.3112971791997552) - rollout_corr/rollout_log_ppl:np.float64(0.2666768589988351) - rollout_corr/log_ppl_diff:np.float64(0.0011522960849106312) - rollout_corr/log_ppl_abs_diff:np.float64(0.002510662190616131) - rollout_corr/log_ppl_diff_max:np.float64(0.009669288992881775) - rollout_corr/log_ppl_diff_min:np.float64(-0.007052525877952576) - rollout_corr/ppl_ratio:np.float64(1.0011574435047805) - rollout_corr/chi2_token:np.float64(0.002051107119768858) - rollout_corr/chi2_seq:np.float64(0.5519429405685514) - actor/pg_loss:np.float64(0.003786680055782199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008521524032403249) - actor/ppo_kl:np.float64(0.0003417033792416646) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(148988.88128135214) - self_distillation/token_reweight_w_std:np.float64(1866554.9818183868) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014604937750846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08368598294327967) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4603109806776047) - perf/mfu/actor:np.float64(0.03142972200463806) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.4005994796753) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/reward/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/score/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/score/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04617062732616158) - val-core/sciknoweval/acc/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/acc/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07287959815596419) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8152857142857142) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0635348044707675) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6967476190476191) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000639524485450238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000639524485450238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.8828125 - response_length/max:611.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.8828125 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00021154806017875671 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3727056793868542) - timing_s/agent_loop/generate_sequences/max:np.float64(4.906513184309006) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9096029985012137) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.906513184309006) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:563 - timing_s/gen:11.531464690342546 - timing_s/reward:0.05282927677035332 - timing_s/old_log_prob:2.459550989791751 - timing_s/adv:0.5421040039509535 - timing_s/update_actor:15.288631148636341 - timing_s/step:29.9776940792799 - timing_s/testing:64.2218036558479 - timing_s/save_checkpoint:6.659373983740807 - timing_s/stop_profile:5.859695374965668e-05 - timing_per_token_ms/adv:0.00401195959170937 - timing_per_token_ms/gen:0.17200359014263514 - timing_per_token_ms/update_actor:0.11314686837551502 - perf/total_num_tokens:135122 - perf/time_per_step:29.9776940792799 - perf/throughput:563.4272587922054 (TaskRunner pid=2026458) Training Progress: 60%|██████ | 60/100 [43:07<34:24, 51.61s/it] (TaskRunner pid=2026458) step:61 - global_seqlen/min:14736 - global_seqlen/max:19497 - global_seqlen/minmax_diff:4761 - global_seqlen/balanced_min:17532 - global_seqlen/balanced_max:17536 - global_seqlen/mean:17534.5 - actor/entropy:0.2423284649848938 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49669769406318665 - training/rollout_probs_diff_mean:0.005022233817726374 - training/rollout_probs_diff_std:0.013324090279638767 - training/rollout_actor_probs_pearson_corr:0.9983177781105042 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0002062320709229 - rollout_corr/rollout_is_ratio_fraction_high:5.464779314934276e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0985847590491176e-05 - rollout_corr/rollout_is_max:7.0302934646606445 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03978368639945984 - rollout_corr/rollout_is_eff_sample_size:0.998420353539922 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.002474896376952529 - rollout_corr/rollout_is_seq_max:1.0129971504211426 - rollout_corr/rollout_is_seq_min:0.9919136762619019 - rollout_corr/rollout_is_seq_max_deviation:0.012997150421142578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2963423300534487) - rollout_corr/training_log_ppl:np.float64(0.25556481117382646) - rollout_corr/kl:np.float64(0.0009956922927543133) - rollout_corr/k3_kl:np.float64(0.0010460353189500893) - rollout_corr/rollout_ppl:np.float64(1.2950277719646692) - rollout_corr/rollout_log_ppl:np.float64(0.25456911721266806) - rollout_corr/log_ppl_diff:np.float64(0.0009956939611583948) - rollout_corr/log_ppl_abs_diff:np.float64(0.002202652278356254) - rollout_corr/log_ppl_diff_max:np.float64(0.010044902563095093) - rollout_corr/log_ppl_diff_min:np.float64(-0.009525060653686523) - rollout_corr/ppl_ratio:np.float64(1.0009999251924455) - rollout_corr/chi2_token:np.float64(0.0022548860870301723) - rollout_corr/chi2_seq:np.float64(0.8442793434951454) - actor/pg_loss:np.float64(0.0042004447896033525) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007894714767644473) - actor/ppo_kl:np.float64(0.0002806229531095994) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(132276.08868701942) - self_distillation/token_reweight_w_std:np.float64(1859101.6540320972) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012411118950695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08409801000379957) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4940867945551872) - perf/mfu/actor:np.float64(0.03230972888329671) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0987777709961) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0024950134102255106 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0024950134102255106 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:285.921875 - response_length/max:685.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:285.921875 - response_length_non_aborted/max:685.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.605738937854767e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.467216296121478) - timing_s/agent_loop/generate_sequences/max:np.float64(5.52247816324234) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0488981239686836) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.52247816324234) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:652 - timing_s/gen:11.610483139753342 - timing_s/reward:0.05246041156351566 - timing_s/old_log_prob:2.416056863963604 - timing_s/adv:0.5235678162425756 - timing_s/update_actor:15.423095140606165 - timing_s/step:30.12737883068621 - timing_s/stop_profile:0.00011511333286762238 - timing_per_token_ms/adv:0.003732411932494337 - timing_per_token_ms/gen:0.15862182550622086 - timing_per_token_ms/update_actor:0.10994821024698569 - perf/total_num_tokens:140276 - perf/time_per_step:30.12737883068621 - perf/throughput:582.0121325038822 (TaskRunner pid=2026458) Training Progress: 61%|██████ | 61/100 [43:37<29:22, 45.18s/it] (TaskRunner pid=2026458) step:62 - global_seqlen/min:15907 - global_seqlen/max:20488 - global_seqlen/minmax_diff:4581 - global_seqlen/balanced_min:18196 - global_seqlen/balanced_max:18198 - global_seqlen/mean:18197.0 - actor/entropy:0.26120486855506897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35323476791381836 - training/rollout_probs_diff_mean:0.005538288038223982 - training/rollout_probs_diff_std:0.014037296175956726 - training/rollout_actor_probs_pearson_corr:0.9981968998908997 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9998036623001099 - rollout_corr/rollout_is_ratio_fraction_high:2.8998956622672267e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014499477401841432 - rollout_corr/rollout_is_max:2.314438819885254 - rollout_corr/rollout_is_min:0.32504311203956604 - rollout_corr/rollout_is_std:0.04294585436582565 - rollout_corr/rollout_is_eff_sample_size:0.9981582768191568 - rollout_corr/rollout_is_seq_mean:0.9998564720153809 - rollout_corr/rollout_is_seq_std:0.0027946694754064083 - rollout_corr/rollout_is_seq_max:1.0084697008132935 - rollout_corr/rollout_is_seq_min:0.9910843372344971 - rollout_corr/rollout_is_seq_max_deviation:0.00891566276550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.316816312726587) - rollout_corr/training_log_ppl:np.float64(0.27135412095231004) - rollout_corr/kl:np.float64(0.0013785573038145316) - rollout_corr/k3_kl:np.float64(0.0011328787384172756) - rollout_corr/rollout_ppl:np.float64(1.3149180496111512) - rollout_corr/rollout_log_ppl:np.float64(0.2699755621724762) - rollout_corr/log_ppl_diff:np.float64(0.0013785587798338383) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027030363271478564) - rollout_corr/log_ppl_diff_max:np.float64(0.010782003402709961) - rollout_corr/log_ppl_diff_min:np.float64(-0.008555859327316284) - rollout_corr/ppl_ratio:np.float64(1.0013843621127307) - rollout_corr/chi2_token:np.float64(0.0016965712420642376) - rollout_corr/chi2_seq:np.float64(0.6767998114228249) - actor/pg_loss:np.float64(0.003519455436617136) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008139853748616588) - actor/ppo_kl:np.float64(0.0002313825499626887) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(140065.4251762007) - self_distillation/token_reweight_w_std:np.float64(1931525.736894751) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999061315320432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07281633446109481) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.4214628040790558) - perf/mfu/actor:np.float64(0.033835202551722315) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0380401611328) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003894922323524952 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003894922323524952 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:269.40625 - response_length/max:578.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.40625 - response_length_non_aborted/max:578.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499168574810028 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3069061245769262) - timing_s/agent_loop/generate_sequences/max:np.float64(4.653931180015206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8548159639394726) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.653931180015206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:578 - timing_s/gen:10.786067547276616 - timing_s/reward:0.05438114143908024 - timing_s/old_log_prob:2.2739491928368807 - timing_s/adv:0.471931716427207 - timing_s/update_actor:15.285212887451053 - timing_s/step:28.96465684659779 - timing_s/stop_profile:0.00012806802988052368 - timing_per_token_ms/adv:0.003241823627707912 - timing_per_token_ms/gen:0.15639234931093574 - timing_per_token_ms/update_actor:0.10499816513333965 - perf/total_num_tokens:145576 - perf/time_per_step:28.96465684659779 - perf/throughput:628.2484234622456 (TaskRunner pid=2026458) Training Progress: 62%|██████▏ | 62/100 [44:06<25:32, 40.33s/it] (TaskRunner pid=2026458) step:63 - global_seqlen/min:15346 - global_seqlen/max:20074 - global_seqlen/minmax_diff:4728 - global_seqlen/balanced_min:17670 - global_seqlen/balanced_max:17672 - global_seqlen/mean:17671.0 - actor/entropy:0.2494240403175354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44295963644981384 - training/rollout_probs_diff_mean:0.005514265038073063 - training/rollout_probs_diff_std:0.013980090618133545 - training/rollout_actor_probs_pearson_corr:0.9981726408004761 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9999319911003113 - rollout_corr/rollout_is_ratio_fraction_high:1.4685150745208375e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.811090083327144e-05 - rollout_corr/rollout_is_max:2.1109066009521484 - rollout_corr/rollout_is_min:0.21203994750976562 - rollout_corr/rollout_is_std:0.04199644550681114 - rollout_corr/rollout_is_eff_sample_size:0.9982390473577023 - rollout_corr/rollout_is_seq_mean:0.9999494552612305 - rollout_corr/rollout_is_seq_std:0.003092808648943901 - rollout_corr/rollout_is_seq_max:1.0130763053894043 - rollout_corr/rollout_is_seq_min:0.9914883375167847 - rollout_corr/rollout_is_seq_max_deviation:0.013076305389404297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.302847915329039) - rollout_corr/training_log_ppl:np.float64(0.26041782830725424) - rollout_corr/kl:np.float64(0.001262190604840896) - rollout_corr/k3_kl:np.float64(0.0011403532549252304) - rollout_corr/rollout_ppl:np.float64(1.3011768972501159) - rollout_corr/rollout_log_ppl:np.float64(0.2591556366533041) - rollout_corr/log_ppl_diff:np.float64(0.0012621916539501399) - rollout_corr/log_ppl_abs_diff:np.float64(0.002628726389957592) - rollout_corr/log_ppl_diff_max:np.float64(0.010536238551139832) - rollout_corr/log_ppl_diff_min:np.float64(-0.009842857718467712) - rollout_corr/ppl_ratio:np.float64(1.0012680946383625) - rollout_corr/chi2_token:np.float64(0.0019653229974210262) - rollout_corr/chi2_seq:np.float64(0.7706008821260184) - actor/pg_loss:np.float64(0.0034944345243275166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000709521887529263) - actor/ppo_kl:np.float64(0.00027179806370725146) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(168796.3436012969) - self_distillation/token_reweight_w_std:np.float64(2124951.2078802614) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015958338044584) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0751116150058806) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4171854108572006) - perf/mfu/actor:np.float64(0.03263190662040404) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89733123779297) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005943814758211374 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005943814758211374 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:266.0 - response_length/max:610.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.0 - response_length_non_aborted/max:610.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011231563985347748 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2275551408529282) - timing_s/agent_loop/generate_sequences/max:np.float64(4.99046435020864) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8069054595180205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.99046435020864) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:231 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:11.166345795616508 - timing_s/reward:0.05529453977942467 - timing_s/old_log_prob:2.2847698368132114 - timing_s/adv:0.49184093065559864 - timing_s/update_actor:15.462059672921896 - timing_s/step:29.548018461093307 - timing_s/stop_profile:0.00011456944048404694 - timing_per_token_ms/adv:0.0034791532076254785 - timing_per_token_ms/gen:0.1639794671583721 - timing_per_token_ms/update_actor:0.10937453789345464 - perf/total_num_tokens:141368 - perf/time_per_step:29.548018461093307 - perf/throughput:598.0434871890951 (TaskRunner pid=2026458) Training Progress: 63%|██████▎ | 63/100 [44:36<22:53, 37.11s/it] (TaskRunner pid=2026458) step:64 - global_seqlen/min:14067 - global_seqlen/max:24197 - global_seqlen/minmax_diff:10130 - global_seqlen/balanced_min:18752 - global_seqlen/balanced_max:25893 - global_seqlen/mean:19646.75 - actor/entropy:0.21532860398292542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8697976469993591 - training/rollout_probs_diff_mean:0.004864232614636421 - training/rollout_probs_diff_std:0.013850908726453781 - training/rollout_actor_probs_pearson_corr:0.9980413317680359 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0002894401550293 - rollout_corr/rollout_is_ratio_fraction_high:6.0805057728430256e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.4322023818967864e-05 - rollout_corr/rollout_is_max:8.297144889831543 - rollout_corr/rollout_is_min:0.23644955456256866 - rollout_corr/rollout_is_std:0.04067561775445938 - rollout_corr/rollout_is_eff_sample_size:0.9983489400333069 - rollout_corr/rollout_is_seq_mean:1.000409722328186 - rollout_corr/rollout_is_seq_std:0.0031241707038134336 - rollout_corr/rollout_is_seq_max:1.0219839811325073 - rollout_corr/rollout_is_seq_min:0.9937897324562073 - rollout_corr/rollout_is_seq_max_deviation:0.021983981132507324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2946171956136823) - rollout_corr/training_log_ppl:np.float64(0.25466993783356884) - rollout_corr/kl:np.float64(0.0008532866019912433) - rollout_corr/k3_kl:np.float64(0.0010788948769118178) - rollout_corr/rollout_ppl:np.float64(1.2935234676115215) - rollout_corr/rollout_log_ppl:np.float64(0.25381665135137155) - rollout_corr/log_ppl_diff:np.float64(0.0008532864821972908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022936501272852183) - rollout_corr/log_ppl_diff_max:np.float64(0.00881144404411316) - rollout_corr/log_ppl_diff_min:np.float64(-0.008881300687789917) - rollout_corr/ppl_ratio:np.float64(1.0008577948901802) - rollout_corr/chi2_token:np.float64(0.003156282240524888) - rollout_corr/chi2_seq:np.float64(3.035388365620747) - actor/pg_loss:np.float64(0.0014677918516099453) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004414845861901995) - actor/ppo_kl:np.float64(0.00026685018261352056) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(81162.51946377661) - self_distillation/token_reweight_w_std:np.float64(1170164.887981981) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010531947482377) - self_distillation/token_reweight_w_clip_frac:np.float64(0.049339942750521004) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.263648372143507) - perf/mfu/actor:np.float64(0.03462126720848994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.74031829833984) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08103185892105103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08103185892105103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.2109375 - response_length/max:8192.0 - response_length/min:110.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:321.2109375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010729953646659851 - timing_s/agent_loop/generate_sequences/min:np.float64(1.288361655548215) - timing_s/agent_loop/generate_sequences/max:np.float64(50.12881679646671) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2828551806887845) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.12881679646671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.03219731338322 - timing_s/reward:0.05905804969370365 - timing_s/old_log_prob:2.5028711035847664 - timing_s/adv:0.508072393015027 - timing_s/update_actor:16.261700792238116 - timing_s/step:75.45720112323761 - timing_s/stop_profile:4.878826439380646e-05 - timing_per_token_ms/adv:0.0032325473234442533 - timing_per_token_ms/gen:0.6814082124940195 - timing_per_token_ms/update_actor:0.10346304600148953 - perf/total_num_tokens:157174 - perf/time_per_step:75.45720112323761 - perf/throughput:260.3694505964075 (TaskRunner pid=2026458) Training Progress: 64%|██████▍ | 64/100 [45:51<29:10, 48.62s/it] (TaskRunner pid=2026458) step:65 - global_seqlen/min:15781 - global_seqlen/max:21398 - global_seqlen/minmax_diff:5617 - global_seqlen/balanced_min:18710 - global_seqlen/balanced_max:18715 - global_seqlen/mean:18712.625 - actor/entropy:0.2451825588941574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43764662742614746 - training/rollout_probs_diff_mean:0.005050553474575281 - training/rollout_probs_diff_std:0.013090502470731735 - training/rollout_actor_probs_pearson_corr:0.9983762502670288 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999321103096008 - rollout_corr/rollout_is_ratio_fraction_high:1.2970000170753337e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8910002331249416e-05 - rollout_corr/rollout_is_max:2.924093246459961 - rollout_corr/rollout_is_min:0.4824545979499817 - rollout_corr/rollout_is_std:0.040282461792230606 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:0.9999394416809082 - rollout_corr/rollout_is_seq_std:0.002558288164436817 - rollout_corr/rollout_is_seq_max:1.0071020126342773 - rollout_corr/rollout_is_seq_min:0.9918232560157776 - rollout_corr/rollout_is_seq_max_deviation:0.008176743984222412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2973304083570838) - rollout_corr/training_log_ppl:np.float64(0.2564336106006522) - rollout_corr/kl:np.float64(0.0012040652124341022) - rollout_corr/k3_kl:np.float64(0.000990912636439134) - rollout_corr/rollout_ppl:np.float64(1.2957068742252886) - rollout_corr/rollout_log_ppl:np.float64(0.2552295460482128) - rollout_corr/log_ppl_diff:np.float64(0.0012040645524393767) - rollout_corr/log_ppl_abs_diff:np.float64(0.002526016061892733) - rollout_corr/log_ppl_diff_max:np.float64(0.011661797761917114) - rollout_corr/log_ppl_diff_min:np.float64(-0.006277412176132202) - rollout_corr/ppl_ratio:np.float64(1.001209291163832) - rollout_corr/chi2_token:np.float64(0.0015423318836838007) - rollout_corr/chi2_seq:np.float64(0.9701050808653235) - actor/pg_loss:np.float64(0.0032792859710752964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007688776604481973) - actor/ppo_kl:np.float64(0.0002768948086071532) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(86791.39557419321) - self_distillation/token_reweight_w_std:np.float64(1084418.8798323106) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982076121959835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08052845095517114) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.3758349269628525) - perf/mfu/actor:np.float64(0.03501136292112507) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.70735931396484) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0052852751687169075 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0052852751687169075 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.17578125 - response_length/max:840.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.17578125 - response_length_non_aborted/max:840.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.040077328681946e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2666476387530565) - timing_s/agent_loop/generate_sequences/max:np.float64(6.11080476641655) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.148740480595734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.11080476641655) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:840 - timing_s/gen:12.000718062743545 - timing_s/reward:0.056409286335110664 - timing_s/old_log_prob:2.2715286798775196 - timing_s/adv:0.546351995319128 - timing_s/update_actor:15.278582841157913 - timing_s/step:30.242278227582574 - timing_s/stop_profile:0.0001621972769498825 - timing_per_token_ms/adv:0.0036496215477460274 - timing_per_token_ms/gen:0.1556493179432633 - timing_per_token_ms/update_actor:0.10206065985636645 - perf/total_num_tokens:149701 - perf/time_per_step:30.242278227582574 - perf/throughput:618.757120716292 (TaskRunner pid=2026458) Training Progress: 65%|██████▌ | 65/100 [46:22<25:09, 43.12s/it] (TaskRunner pid=2026458) step:66 - global_seqlen/min:16986 - global_seqlen/max:21356 - global_seqlen/minmax_diff:4370 - global_seqlen/balanced_min:19034 - global_seqlen/balanced_max:19038 - global_seqlen/mean:19037.375 - actor/entropy:0.2407594919204712 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26579251885414124 - training/rollout_probs_diff_mean:0.005038884002715349 - training/rollout_probs_diff_std:0.013142738491296768 - training/rollout_actor_probs_pearson_corr:0.9983550906181335 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.000045657157898 - rollout_corr/rollout_is_ratio_fraction_high:2.5642670152592473e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.974934462457895e-05 - rollout_corr/rollout_is_max:2.355177402496338 - rollout_corr/rollout_is_min:0.19801293313503265 - rollout_corr/rollout_is_std:0.04063016548752785 - rollout_corr/rollout_is_eff_sample_size:0.9983520292581156 - rollout_corr/rollout_is_seq_mean:1.0001461505889893 - rollout_corr/rollout_is_seq_std:0.0026480352971702814 - rollout_corr/rollout_is_seq_max:1.0091423988342285 - rollout_corr/rollout_is_seq_min:0.9913557171821594 - rollout_corr/rollout_is_seq_max_deviation:0.009142398834228516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3078779838979244) - rollout_corr/training_log_ppl:np.float64(0.26354357780655846) - rollout_corr/kl:np.float64(0.0010749662497779866) - rollout_corr/k3_kl:np.float64(0.0014432408553943787) - rollout_corr/rollout_ppl:np.float64(1.3064235104247928) - rollout_corr/rollout_log_ppl:np.float64(0.2624686111230403) - rollout_corr/log_ppl_diff:np.float64(0.0010749666835181415) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029108942835591733) - rollout_corr/log_ppl_diff_max:np.float64(0.013920098543167114) - rollout_corr/log_ppl_diff_min:np.float64(-0.009019121527671814) - rollout_corr/ppl_ratio:np.float64(1.0010822340846062) - rollout_corr/chi2_token:np.float64(0.003545147832483053) - rollout_corr/chi2_seq:np.float64(1.548627549316734) - actor/pg_loss:np.float64(0.003003176534548402) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009145650715254305) - actor/ppo_kl:np.float64(0.0003188287759954278) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(126959.81397781824) - self_distillation/token_reweight_w_std:np.float64(1629415.2238851967) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008254635613412) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06369021107093431) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.4391012266278267) - perf/mfu/actor:np.float64(0.03551272812953314) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.41056060791016) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004540354944765568 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004540354944765568 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:304.66796875 - response_length/max:841.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:304.66796875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014583580195903778 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5362531188875437) - timing_s/agent_loop/generate_sequences/max:np.float64(6.417147822678089) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.267450157953135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.417147822678089) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.298093724995852 - timing_s/reward:0.05658415146172047 - timing_s/old_log_prob:2.3751799557358027 - timing_s/adv:0.4913473054766655 - timing_s/update_actor:15.358580730855465 - timing_s/step:30.66549047641456 - timing_s/stop_profile:0.0001288391649723053 - timing_per_token_ms/adv:0.0032262017838374873 - timing_per_token_ms/gen:0.1576779758317309 - timing_per_token_ms/update_actor:0.10084492170569383 - perf/total_num_tokens:152299 - perf/time_per_step:30.66549047641456 - perf/throughput:620.8077778714162 (TaskRunner pid=2026458) Training Progress: 66%|██████▌ | 66/100 [46:52<22:19, 39.40s/it] (TaskRunner pid=2026458) step:67 - global_seqlen/min:15159 - global_seqlen/max:19915 - global_seqlen/minmax_diff:4756 - global_seqlen/balanced_min:18046 - global_seqlen/balanced_max:18050 - global_seqlen/mean:18048.5 - actor/entropy:0.22801370918750763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9852267503738403 - training/rollout_probs_diff_mean:0.005018836818635464 - training/rollout_probs_diff_std:0.014277998358011246 - training/rollout_actor_probs_pearson_corr:0.9979830384254456 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0002268552780151 - rollout_corr/rollout_is_ratio_fraction_high:2.8704287615255453e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001004650112008676 - rollout_corr/rollout_is_max:2.3428878784179688 - rollout_corr/rollout_is_min:0.0007197936065495014 - rollout_corr/rollout_is_std:0.040458161383867264 - rollout_corr/rollout_is_eff_sample_size:0.9983666439269478 - rollout_corr/rollout_is_seq_mean:1.0003077983856201 - rollout_corr/rollout_is_seq_std:0.0026670496445149183 - rollout_corr/rollout_is_seq_max:1.0164320468902588 - rollout_corr/rollout_is_seq_min:0.9921717047691345 - rollout_corr/rollout_is_seq_max_deviation:0.01643204689025879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2767656035721302) - rollout_corr/training_log_ppl:np.float64(0.24004710461304057) - rollout_corr/kl:np.float64(0.0011609104617287258) - rollout_corr/k3_kl:np.float64(0.001655576513314827) - rollout_corr/rollout_ppl:np.float64(1.2751760962419212) - rollout_corr/rollout_log_ppl:np.float64(0.23888619328499772) - rollout_corr/log_ppl_diff:np.float64(0.0011609113280428573) - rollout_corr/log_ppl_abs_diff:np.float64(0.003236807693610899) - rollout_corr/log_ppl_diff_max:np.float64(0.017100274562835693) - rollout_corr/log_ppl_diff_min:np.float64(-0.02350728213787079) - rollout_corr/ppl_ratio:np.float64(1.0011714671272784) - rollout_corr/chi2_token:np.float64(0.004160782555118203) - rollout_corr/chi2_seq:np.float64(5.24587120115757) - actor/pg_loss:np.float64(0.0036287190159782767) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001126493662923167) - actor/ppo_kl:np.float64(0.0005246336532138685) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(249329.88290784857) - self_distillation/token_reweight_w_std:np.float64(2866482.3437989554) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994784640148282) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06014655635226518) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.471420306712389) - perf/mfu/actor:np.float64(0.033855896624554245) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.65926361083984) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004662652965635061 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004662652965635061 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:272.171875 - response_length/max:628.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.171875 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.801262497901917e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2853057831525803) - timing_s/agent_loop/generate_sequences/max:np.float64(5.174439303576946) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.863578474461974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.174439303576946) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:351 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.778497187420726 - timing_s/reward:0.052504874765872955 - timing_s/old_log_prob:2.3092593122273684 - timing_s/adv:0.5003343336284161 - timing_s/update_actor:15.202262729406357 - timing_s/step:28.936630526557565 - timing_s/stop_profile:0.00012007169425487518 - timing_per_token_ms/adv:0.0034652071753083088 - timing_per_token_ms/gen:0.1546945460046605 - timing_per_token_ms/update_actor:0.10528757742614592 - perf/total_num_tokens:144388 - perf/time_per_step:28.936630526557565 - perf/throughput:623.7250043136634 (TaskRunner pid=2026458) Training Progress: 67%|██████▋ | 67/100 [47:21<19:57, 36.28s/it] (TaskRunner pid=2026458) step:68 - global_seqlen/min:14735 - global_seqlen/max:19627 - global_seqlen/minmax_diff:4892 - global_seqlen/balanced_min:17371 - global_seqlen/balanced_max:17375 - global_seqlen/mean:17373.75 - actor/entropy:0.26004546880722046 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23114943504333496 - training/rollout_probs_diff_mean:0.005509829614311457 - training/rollout_probs_diff_std:0.013461560010910034 - training/rollout_actor_probs_pearson_corr:0.9983240365982056 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:1.0001424551010132 - rollout_corr/rollout_is_ratio_fraction_high:3.430884680710733e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.14632738486398e-05 - rollout_corr/rollout_is_max:2.094249725341797 - rollout_corr/rollout_is_min:0.4616723358631134 - rollout_corr/rollout_is_std:0.04154840111732483 - rollout_corr/rollout_is_eff_sample_size:0.998277299143981 - rollout_corr/rollout_is_seq_mean:1.0000828504562378 - rollout_corr/rollout_is_seq_std:0.0031394551042467356 - rollout_corr/rollout_is_seq_max:1.0125868320465088 - rollout_corr/rollout_is_seq_min:0.9910873770713806 - rollout_corr/rollout_is_seq_max_deviation:0.012586832046508789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3149044853635132) - rollout_corr/training_log_ppl:np.float64(0.26928867847891524) - rollout_corr/kl:np.float64(0.0007745910923624422) - rollout_corr/k3_kl:np.float64(0.0011792316492460486) - rollout_corr/rollout_ppl:np.float64(1.3139065788127482) - rollout_corr/rollout_log_ppl:np.float64(0.26851408526999876) - rollout_corr/log_ppl_diff:np.float64(0.0007745932089164853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025513150612823665) - rollout_corr/log_ppl_diff_max:np.float64(0.009900659322738647) - rollout_corr/log_ppl_diff_min:np.float64(-0.015614360570907593) - rollout_corr/ppl_ratio:np.float64(1.0007805349305272) - rollout_corr/chi2_token:np.float64(0.0031827117782086134) - rollout_corr/chi2_seq:np.float64(0.9155827462673187) - actor/pg_loss:np.float64(0.0037549110129475594) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007232226507767336) - actor/ppo_kl:np.float64(-4.5092836657545377e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(174603.2610843731) - self_distillation/token_reweight_w_std:np.float64(2132750.0757954237) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986913346219808) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0688194558606483) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.4323424845933914) - perf/mfu/actor:np.float64(0.03227397909516918) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.99407958984375) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006370724178850651 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006370724178850651 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.7109375 - response_length/max:542.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.7109375 - response_length_non_aborted/max:542.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010382384061813354 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2069221679121256) - timing_s/agent_loop/generate_sequences/max:np.float64(4.399640990421176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.518257544754306) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.399640990421176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:508 - timing_s/agent_loop/slowest/response_length:542 - timing_s/gen:10.388320125639439 - timing_s/reward:0.053288890048861504 - timing_s/old_log_prob:2.246592290699482 - timing_s/adv:0.4770530089735985 - timing_s/update_actor:15.537012735381722 - timing_s/step:28.787579828873277 - timing_s/stop_profile:0.00013300031423568726 - timing_per_token_ms/adv:0.003432282962613127 - timing_per_token_ms/gen:0.1782056493916945 - timing_per_token_ms/update_actor:0.1117851121331155 - perf/total_num_tokens:138990 - perf/time_per_step:28.787579828873277 - perf/throughput:603.5154779692363 (TaskRunner pid=2026458) Training Progress: 68%|██████▊ | 68/100 [47:50<18:09, 34.04s/it] (TaskRunner pid=2026458) step:69 - global_seqlen/min:15072 - global_seqlen/max:24554 - global_seqlen/minmax_diff:9482 - global_seqlen/balanced_min:18606 - global_seqlen/balanced_max:18614 - global_seqlen/mean:18608.375 - actor/entropy:0.24838018417358398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21835267543792725 - training/rollout_probs_diff_mean:0.004963765386492014 - training/rollout_probs_diff_std:0.012902619317173958 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9996421337127686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.750844502472319e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.26981014013290405 - rollout_corr/rollout_is_std:0.039976488798856735 - rollout_corr/rollout_is_eff_sample_size:0.9984033607144375 - rollout_corr/rollout_is_seq_mean:0.9996398687362671 - rollout_corr/rollout_is_seq_std:0.0026584318839013577 - rollout_corr/rollout_is_seq_max:1.0071197748184204 - rollout_corr/rollout_is_seq_min:0.9907388687133789 - rollout_corr/rollout_is_seq_max_deviation:0.009261131286621094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3142020958475769) - rollout_corr/training_log_ppl:np.float64(0.2681919914903119) - rollout_corr/kl:np.float64(0.0011005508938133346) - rollout_corr/k3_kl:np.float64(0.0010078800825397138) - rollout_corr/rollout_ppl:np.float64(1.3127118106931448) - rollout_corr/rollout_log_ppl:np.float64(0.2670914397749584) - rollout_corr/log_ppl_diff:np.float64(0.0011005517153535038) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023552981147076935) - rollout_corr/log_ppl_diff_max:np.float64(0.013139069080352783) - rollout_corr/log_ppl_diff_min:np.float64(-0.007561713457107544) - rollout_corr/ppl_ratio:np.float64(1.001105590723455) - rollout_corr/chi2_token:np.float64(0.0017845977563410997) - rollout_corr/chi2_seq:np.float64(0.37655540159903467) - actor/pg_loss:np.float64(0.004595633479766548) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005241842873147107) - actor/ppo_kl:np.float64(-0.0001359343073588093) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(242376.49935866683) - self_distillation/token_reweight_w_std:np.float64(3279716.472702818) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019234952051193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08817204824299552) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.47684382647275925) - perf/mfu/actor:np.float64(0.03472723156176391) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89558792114258) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034019839949905872 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034019839949905872 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.69921875 - response_length/max:979.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.69921875 - response_length_non_aborted/max:979.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001641698181629181 - timing_s/agent_loop/generate_sequences/min:np.float64(1.224152859300375) - timing_s/agent_loop/generate_sequences/max:np.float64(6.517763238400221) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8646460080635734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.517763238400221) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:979 - timing_s/gen:12.344549676403403 - timing_s/reward:0.0557593759149313 - timing_s/old_log_prob:2.2437479849904776 - timing_s/adv:0.49268065579235554 - timing_s/update_actor:14.758547224104404 - timing_s/step:29.983076663687825 - timing_s/stop_profile:0.00012305378913879395 - timing_per_token_ms/adv:0.003309535731843562 - timing_per_token_ms/gen:0.17747896882184463 - timing_per_token_ms/update_actor:0.09913914584229147 - perf/total_num_tokens:148867 - perf/time_per_step:29.983076663687825 - perf/throughput:620.6292705957158 (TaskRunner pid=2026458) Training Progress: 69%|██████▉ | 69/100 [48:20<16:58, 32.84s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". To determine the correct answer, we need to analyze the structure of the product and compare it with the options. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrile group (–n) attached to a cyclohexyl ring with a chlorine atom and a trifluoromethyl group (–C(F)(F)F). The structure also includes a benzene ring with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A: Contains a nitrile group and a cyclohexyl ring with a chlorine atom and a trifluoromethyl group, matching the product's structure. (TaskRunner pid=2026458) - Option B: Contains a nitrile group but does not have the correct trifluoromethyl group or the correct chlorine atom placement. (TaskRunner pid=2026458) - Option C: Contains a nitrile group but does not match the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) - Option D: Contains a nitrile group but does not have the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2026458) step:70 - global_seqlen/min:12837 - global_seqlen/max:19669 - global_seqlen/minmax_diff:6832 - global_seqlen/balanced_min:17821 - global_seqlen/balanced_max:17831 - global_seqlen/mean:17825.125 - actor/entropy:0.2367510199546814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33400535583496094 - training/rollout_probs_diff_mean:0.005260511301457882 - training/rollout_probs_diff_std:0.013495014049112797 - training/rollout_actor_probs_pearson_corr:0.9982122778892517 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9999286532402039 - rollout_corr/rollout_is_ratio_fraction_high:4.527823330136016e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5092745343281422e-05 - rollout_corr/rollout_is_max:2.0651345252990723 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.04225678741931915 - rollout_corr/rollout_is_eff_sample_size:0.9982174280795298 - rollout_corr/rollout_is_seq_mean:0.9999866485595703 - rollout_corr/rollout_is_seq_std:0.00303856679238379 - rollout_corr/rollout_is_seq_max:1.0101226568222046 - rollout_corr/rollout_is_seq_min:0.9901509284973145 - rollout_corr/rollout_is_seq_max_deviation:0.01012265682220459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2921742992475629) - rollout_corr/training_log_ppl:np.float64(0.2528939112526132) - rollout_corr/kl:np.float64(0.0014884298633148774) - rollout_corr/k3_kl:np.float64(0.0010952503159273874) - rollout_corr/rollout_ppl:np.float64(1.2902372791431844) - rollout_corr/rollout_log_ppl:np.float64(0.2514054805797059) - rollout_corr/log_ppl_diff:np.float64(0.0014884306729072705) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028520045889308676) - rollout_corr/log_ppl_diff_max:np.float64(0.0136488676071167) - rollout_corr/log_ppl_diff_min:np.float64(-0.008753955364227295) - rollout_corr/ppl_ratio:np.float64(1.0014950232580304) - rollout_corr/chi2_token:np.float64(0.0014098831452429295) - rollout_corr/chi2_seq:np.float64(0.360071137547493) - actor/pg_loss:np.float64(0.0027717185439541936) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434370364630013) - actor/ppo_kl:np.float64(0.0005119445677905787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(69071.25273821992) - self_distillation/token_reweight_w_std:np.float64(1045658.8316577639) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004556903149933) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06493389664683491) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.3887019380927086) - perf/mfu/actor:np.float64(0.0325070599234879) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.87537002563477) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/score/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/score/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/score/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0541181009825183) - val-core/sciknoweval/acc/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07272307414099477) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8114142857142858) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06774926114873739) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.712047619047619) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036373515613377094 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036373515613377094 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.81640625 - response_length/max:938.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.81640625 - response_length_non_aborted/max:938.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016323477029800415 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2699729204177856) - timing_s/agent_loop/generate_sequences/max:np.float64(6.416786439716816) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.761028841654479) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.416786439716816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:938 - timing_s/gen:12.333267584443092 - timing_s/reward:0.05591795593500137 - timing_s/old_log_prob:2.2401132248342037 - timing_s/adv:0.506872920319438 - timing_s/update_actor:15.221702938899398 - timing_s/step:30.44651348516345 - timing_s/testing:81.57863584533334 - timing_s/save_checkpoint:6.6424451023340225 - timing_s/stop_profile:0.00010693445801734924 - timing_per_token_ms/adv:0.003554483631387143 - timing_per_token_ms/gen:0.18614286165149482 - timing_per_token_ms/update_actor:0.1067433113295096 - perf/total_num_tokens:142601 - perf/time_per_step:30.44651348516345 - perf/throughput:585.4570182128132 (TaskRunner pid=2026458) Training Progress: 70%|███████ | 70/100 [50:19<29:18, 58.61s/it] (TaskRunner pid=2026458) step:71 - global_seqlen/min:15483 - global_seqlen/max:28966 - global_seqlen/minmax_diff:13483 - global_seqlen/balanced_min:18380 - global_seqlen/balanced_max:25891 - global_seqlen/mean:19333.5 - actor/entropy:0.21173985302448273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4046512842178345 - training/rollout_probs_diff_mean:0.0045140874572098255 - training/rollout_probs_diff_std:0.012615382671356201 - training/rollout_actor_probs_pearson_corr:0.9983184337615967 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998964667320251 - rollout_corr/rollout_is_ratio_fraction_high:1.2943308320245706e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.471654342021793e-05 - rollout_corr/rollout_is_max:2.0498809814453125 - rollout_corr/rollout_is_min:0.38866347074508667 - rollout_corr/rollout_is_std:0.03843024745583534 - rollout_corr/rollout_is_eff_sample_size:0.9985250563653204 - rollout_corr/rollout_is_seq_mean:0.9998904466629028 - rollout_corr/rollout_is_seq_std:0.0028965510427951813 - rollout_corr/rollout_is_seq_max:1.0107746124267578 - rollout_corr/rollout_is_seq_min:0.9922298192977905 - rollout_corr/rollout_is_seq_max_deviation:0.010774612426757812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2945193266496062) - rollout_corr/training_log_ppl:np.float64(0.2542406297643538) - rollout_corr/kl:np.float64(0.0010824492369110317) - rollout_corr/k3_kl:np.float64(0.0011051825894128342) - rollout_corr/rollout_ppl:np.float64(1.293056899216026) - rollout_corr/rollout_log_ppl:np.float64(0.2531581802359142) - rollout_corr/log_ppl_diff:np.float64(0.0010824495284396107) - rollout_corr/log_ppl_abs_diff:np.float64(0.00271785432141769) - rollout_corr/log_ppl_diff_max:np.float64(0.014131128787994385) - rollout_corr/log_ppl_diff_min:np.float64(-0.009531468152999878) - rollout_corr/ppl_ratio:np.float64(1.0010888520628214) - rollout_corr/chi2_token:np.float64(0.002212358172982931) - rollout_corr/chi2_seq:np.float64(0.6793141439557076) - actor/pg_loss:np.float64(0.002975177369080484) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000972424295014207) - actor/ppo_kl:np.float64(7.461081707615946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(134136.12361396686) - self_distillation/token_reweight_w_std:np.float64(1624645.7259825736) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988281566184014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07092063533491455) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.4537533223628998) - perf/mfu/actor:np.float64(0.03374089352858298) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.65079879760742) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06832286715507507 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06832286715507507 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.796875 - response_length/max:8192.0 - response_length/min:93.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:301.796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.677132427692413e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1272850260138512) - timing_s/agent_loop/generate_sequences/max:np.float64(47.67028555460274) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.04078216401831) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(47.67028555460274) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:666 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:53.34413158334792 - timing_s/reward:0.05393144115805626 - timing_s/old_log_prob:2.626548172906041 - timing_s/adv:0.5594039708375931 - timing_s/update_actor:16.17577089741826 - timing_s/step:72.8568678945303 - timing_s/stop_profile:2.8194859623908997e-05 - timing_per_token_ms/adv:0.0036168048389944468 - timing_per_token_ms/gen:0.6904495415913527 - timing_per_token_ms/update_actor:0.1045838240451694 - perf/total_num_tokens:154668 - perf/time_per_step:72.8568678945303 - perf/throughput:265.3627661840711 (TaskRunner pid=2026458) Training Progress: 71%|███████ | 71/100 [51:32<30:23, 62.89s/it] (TaskRunner pid=2026458) step:72 - global_seqlen/min:14180 - global_seqlen/max:19404 - global_seqlen/minmax_diff:5224 - global_seqlen/balanced_min:16784 - global_seqlen/balanced_max:16786 - global_seqlen/mean:16785.25 - actor/entropy:0.2532234191894531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37658625841140747 - training/rollout_probs_diff_mean:0.005358405876904726 - training/rollout_probs_diff_std:0.01324445940554142 - training/rollout_actor_probs_pearson_corr:0.9983224272727966 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.000357747077942 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6467605829238892 - rollout_corr/rollout_is_min:0.5746954679489136 - rollout_corr/rollout_is_std:0.04072979837656021 - rollout_corr/rollout_is_eff_sample_size:0.9983450191216581 - rollout_corr/rollout_is_seq_mean:1.0003376007080078 - rollout_corr/rollout_is_seq_std:0.0029273421969264746 - rollout_corr/rollout_is_seq_max:1.0102252960205078 - rollout_corr/rollout_is_seq_min:0.9919271469116211 - rollout_corr/rollout_is_seq_max_deviation:0.010225296020507812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3001699862070382) - rollout_corr/training_log_ppl:np.float64(0.25781238308991306) - rollout_corr/kl:np.float64(0.0006664992815785808) - rollout_corr/k3_kl:np.float64(0.0009609586412011595) - rollout_corr/rollout_ppl:np.float64(1.2992715733125806) - rollout_corr/rollout_log_ppl:np.float64(0.25714588271512184) - rollout_corr/log_ppl_diff:np.float64(0.0006665003747912124) - rollout_corr/log_ppl_abs_diff:np.float64(0.002382488935836591) - rollout_corr/log_ppl_diff_max:np.float64(0.009035348892211914) - rollout_corr/log_ppl_diff_min:np.float64(-0.009852290153503418) - rollout_corr/ppl_ratio:np.float64(1.0006713322363794) - rollout_corr/chi2_token:np.float64(0.0025634136982262135) - rollout_corr/chi2_seq:np.float64(1.15624994575046) - actor/pg_loss:np.float64(0.003654895001091063) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009709688329166966) - actor/ppo_kl:np.float64(0.00014433233151223135) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(213152.86371882842) - self_distillation/token_reweight_w_std:np.float64(2646677.052918058) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9974161058198661) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09049051211331971) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.43892450630664825) - perf/mfu/actor:np.float64(0.03149296183920366) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40920639038086) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009050944820046425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009050944820046425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.6953125 - response_length/max:467.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:237.6953125 - response_length_non_aborted/max:467.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.147497773170471e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3375506270676851) - timing_s/agent_loop/generate_sequences/max:np.float64(4.0367792677134275) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.622172743664123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.0367792677134275) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:467 - timing_s/gen:10.345969874411821 - timing_s/reward:0.05066956579685211 - timing_s/old_log_prob:2.2474361546337605 - timing_s/adv:0.4828864671289921 - timing_s/update_actor:14.702434957027435 - timing_s/step:27.925694411620498 - timing_s/stop_profile:3.0016526579856873e-05 - timing_per_token_ms/adv:0.0035960625186472655 - timing_per_token_ms/gen:0.17002415570109813 - timing_per_token_ms/update_actor:0.1094892461910564 - perf/total_num_tokens:134282 - perf/time_per_step:27.925694411620498 - perf/throughput:601.0683119491305 (TaskRunner pid=2026458) Training Progress: 72%|███████▏ | 72/100 [52:00<24:27, 52.41s/it] (TaskRunner pid=2026458) step:73 - global_seqlen/min:15559 - global_seqlen/max:20190 - global_seqlen/minmax_diff:4631 - global_seqlen/balanced_min:17173 - global_seqlen/balanced_max:17174 - global_seqlen/mean:17173.5 - actor/entropy:0.24391600489616394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135395884513855 - training/rollout_probs_diff_mean:0.0049339784309268 - training/rollout_probs_diff_std:0.012776896357536316 - training/rollout_actor_probs_pearson_corr:0.998493492603302 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000167965888977 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.0463656003121287e-05 - rollout_corr/rollout_is_max:1.767087697982788 - rollout_corr/rollout_is_min:0.47380420565605164 - rollout_corr/rollout_is_std:0.03922998905181885 - rollout_corr/rollout_is_eff_sample_size:0.9984639671449654 - rollout_corr/rollout_is_seq_mean:1.0002386569976807 - rollout_corr/rollout_is_seq_std:0.0029904802795499563 - rollout_corr/rollout_is_seq_max:1.0164623260498047 - rollout_corr/rollout_is_seq_min:0.9890998005867004 - rollout_corr/rollout_is_seq_max_deviation:0.016462326049804688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2968735643662512) - rollout_corr/training_log_ppl:np.float64(0.25511286556138657) - rollout_corr/kl:np.float64(0.0009459122673263209) - rollout_corr/k3_kl:np.float64(0.0009600993125502555) - rollout_corr/rollout_ppl:np.float64(1.2955679851584136) - rollout_corr/rollout_log_ppl:np.float64(0.2541669542551972) - rollout_corr/log_ppl_diff:np.float64(0.0009459113061893731) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024720369547139853) - rollout_corr/log_ppl_diff_max:np.float64(0.012520670890808105) - rollout_corr/log_ppl_diff_min:np.float64(-0.0084361732006073) - rollout_corr/ppl_ratio:np.float64(1.0009514002595097) - rollout_corr/chi2_token:np.float64(0.001954789971932769) - rollout_corr/chi2_seq:np.float64(0.6583140406291932) - actor/pg_loss:np.float64(0.0032611776841804385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000914203752472531) - actor/ppo_kl:np.float64(0.00038008014968227144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(311191.56539016124) - self_distillation/token_reweight_w_std:np.float64(2860058.778589467) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015116163995117) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06564340484328568) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.42411575466394424) - perf/mfu/actor:np.float64(0.031904336867600726) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.16930770874023) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001247106003575027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001247106003575027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.453125 - response_length/max:556.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.453125 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.804398536682129e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1476398799568415) - timing_s/agent_loop/generate_sequences/max:np.float64(4.690207960084081) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7287142683780985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.690207960084081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.320297587662935 - timing_s/reward:0.0441241730004549 - timing_s/old_log_prob:2.164985027164221 - timing_s/adv:0.4551666658371687 - timing_s/update_actor:14.973258456215262 - timing_s/step:27.992081409320235 - timing_s/stop_profile:2.880394458770752e-05 - timing_per_token_ms/adv:0.003313001614676454 - timing_per_token_ms/gen:0.15719700218824917 - timing_per_token_ms/update_actor:0.10898519853418975 - perf/total_num_tokens:137388 - perf/time_per_step:27.992081409320235 - perf/throughput:613.5127913096851 (TaskRunner pid=2026458) Training Progress: 73%|███████▎ | 73/100 [52:28<20:17, 45.09s/it] (TaskRunner pid=2026458) step:74 - global_seqlen/min:13447 - global_seqlen/max:21573 - global_seqlen/minmax_diff:8126 - global_seqlen/balanced_min:16831 - global_seqlen/balanced_max:16917 - global_seqlen/mean:16843.875 - actor/entropy:0.24996380507946014 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424726128578186 - training/rollout_probs_diff_mean:0.004949183203279972 - training/rollout_probs_diff_std:0.012561521492898464 - training/rollout_actor_probs_pearson_corr:0.9985408782958984 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998319149017334 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.6325607399921864e-05 - rollout_corr/rollout_is_max:1.7145717144012451 - rollout_corr/rollout_is_min:0.17540410161018372 - rollout_corr/rollout_is_std:0.03882066160440445 - rollout_corr/rollout_is_eff_sample_size:0.9984948673727597 - rollout_corr/rollout_is_seq_mean:0.9999223947525024 - rollout_corr/rollout_is_seq_std:0.0029323359485715628 - rollout_corr/rollout_is_seq_max:1.009851336479187 - rollout_corr/rollout_is_seq_min:0.9908139109611511 - rollout_corr/rollout_is_seq_max_deviation:0.009851336479187012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3159835655242205) - rollout_corr/training_log_ppl:np.float64(0.26891483918007) - rollout_corr/kl:np.float64(0.0010735450956096404) - rollout_corr/k3_kl:np.float64(0.0011378318484958072) - rollout_corr/rollout_ppl:np.float64(1.3145075496286154) - rollout_corr/rollout_log_ppl:np.float64(0.26784129165753257) - rollout_corr/log_ppl_diff:np.float64(0.0010735475225374103) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030432986677624285) - rollout_corr/log_ppl_diff_max:np.float64(0.012433648109436035) - rollout_corr/log_ppl_diff_min:np.float64(-0.009890466928482056) - rollout_corr/ppl_ratio:np.float64(1.0010810112580657) - rollout_corr/chi2_token:np.float64(0.0024086034391075373) - rollout_corr/chi2_seq:np.float64(1.130532833514735) - actor/pg_loss:np.float64(0.0028035463765263557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007331344913836801) - actor/ppo_kl:np.float64(0.00016281594710676472) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(183593.82668150146) - self_distillation/token_reweight_w_std:np.float64(2093931.331328835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001840919721872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060318807751173154) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3842148035764694) - perf/mfu/actor:np.float64(0.031244242735268143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2487654685974) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016619311645627022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016619311645627022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:252.96484375 - response_length/max:993.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:252.96484375 - response_length_non_aborted/max:993.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7175992727279663e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.240340730175376) - timing_s/agent_loop/generate_sequences/max:np.float64(6.539421858265996) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.619629321692628) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.539421858265996) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:993 - timing_s/gen:12.270534712821245 - timing_s/reward:0.04380444623529911 - timing_s/old_log_prob:2.2239825148135424 - timing_s/adv:0.5528004374355078 - timing_s/update_actor:15.180137295275927 - timing_s/step:30.30796923674643 - timing_s/stop_profile:5.2906572818756104e-05 - timing_per_token_ms/adv:0.004102384675701908 - timing_per_token_ms/gen:0.18947999062402515 - timing_per_token_ms/update_actor:0.11265324409671117 - perf/total_num_tokens:134751 - perf/time_per_step:30.30796923674643 - perf/throughput:555.7572950014052 (TaskRunner pid=2026458) Training Progress: 74%|███████▍ | 74/100 [52:58<17:37, 40.66s/it] (TaskRunner pid=2026458) step:75 - global_seqlen/min:14038 - global_seqlen/max:20039 - global_seqlen/minmax_diff:6001 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18457.625 - actor/entropy:0.24194937944412231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4665491580963135 - training/rollout_probs_diff_mean:0.005109612364321947 - training/rollout_probs_diff_std:0.013375498354434967 - training/rollout_actor_probs_pearson_corr:0.9982732534408569 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9997668862342834 - rollout_corr/rollout_is_ratio_fraction_high:2.9664348403457552e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.449652260518633e-05 - rollout_corr/rollout_is_max:2.278925657272339 - rollout_corr/rollout_is_min:0.3029129207134247 - rollout_corr/rollout_is_std:0.040616225451231 - rollout_corr/rollout_is_eff_sample_size:0.9983523857083623 - rollout_corr/rollout_is_seq_mean:0.9997424483299255 - rollout_corr/rollout_is_seq_std:0.002785229589790106 - rollout_corr/rollout_is_seq_max:1.00969660282135 - rollout_corr/rollout_is_seq_min:0.9920895099639893 - rollout_corr/rollout_is_seq_max_deviation:0.009696602821350098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2928350591100752) - rollout_corr/training_log_ppl:np.float64(0.2525325442838948) - rollout_corr/kl:np.float64(0.0011320576819571215) - rollout_corr/k3_kl:np.float64(0.0010110275231340893) - rollout_corr/rollout_ppl:np.float64(1.2913245973177254) - rollout_corr/rollout_log_ppl:np.float64(0.2514004855765961) - rollout_corr/log_ppl_diff:np.float64(0.001132058707298711) - rollout_corr/log_ppl_abs_diff:np.float64(0.002472092892276123) - rollout_corr/log_ppl_diff_max:np.float64(0.01102399080991745) - rollout_corr/log_ppl_diff_min:np.float64(-0.008037805557250977) - rollout_corr/ppl_ratio:np.float64(1.0011374710593373) - rollout_corr/chi2_token:np.float64(0.0017729024402797222) - rollout_corr/chi2_seq:np.float64(0.6597908593248576) - actor/pg_loss:np.float64(0.0018322557443752885) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036231086232874077) - actor/ppo_kl:np.float64(-3.7132614664869834e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(138490.71220888244) - self_distillation/token_reweight_w_std:np.float64(1204680.9713628318) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000044918153435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041466833936283365) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.29594044387340546) - perf/mfu/actor:np.float64(0.03412545800281451) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2256622314453) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.489075894933194e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:6.489075894933194e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:263.36328125 - response_length/max:680.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.36328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.105005741119385e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.53872056491673) - timing_s/agent_loop/generate_sequences/max:np.float64(5.3331572357565165) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.911040708415385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.3331572357565165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:367 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.2033476177603 - timing_s/reward:0.055844465270638466 - timing_s/old_log_prob:2.219141924753785 - timing_s/adv:0.4940295107662678 - timing_s/update_actor:15.452317954972386 - timing_s/step:29.51427774503827 - timing_s/stop_profile:0.0001329611986875534 - timing_per_token_ms/adv:0.003345700697992481 - timing_per_token_ms/gen:0.16617000070838908 - timing_per_token_ms/update_actor:0.10464725252417623 - perf/total_num_tokens:147661 - perf/time_per_step:29.51427774503827 - perf/throughput:625.3795251046915 (TaskRunner pid=2026458) Training Progress: 75%|███████▌ | 75/100 [53:28<15:33, 37.33s/it] (TaskRunner pid=2026458) step:76 - global_seqlen/min:15400 - global_seqlen/max:20933 - global_seqlen/minmax_diff:5533 - global_seqlen/balanced_min:18318 - global_seqlen/balanced_max:18443 - global_seqlen/mean:18349.25 - actor/entropy:0.23565568029880524 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6839720010757446 - training/rollout_probs_diff_mean:0.00476441252976656 - training/rollout_probs_diff_std:0.01295106578618288 - training/rollout_actor_probs_pearson_corr:0.9984387159347534 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.999829113483429 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.072877345606685e-05 - rollout_corr/rollout_is_max:1.8591279983520508 - rollout_corr/rollout_is_min:0.14236801862716675 - rollout_corr/rollout_is_std:0.038981545716524124 - rollout_corr/rollout_is_eff_sample_size:0.9984822693566601 - rollout_corr/rollout_is_seq_mean:0.9998883008956909 - rollout_corr/rollout_is_seq_std:0.0026900693774223328 - rollout_corr/rollout_is_seq_max:1.0127232074737549 - rollout_corr/rollout_is_seq_min:0.9910237193107605 - rollout_corr/rollout_is_seq_max_deviation:0.012723207473754883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3050027494318783) - rollout_corr/training_log_ppl:np.float64(0.26096004332066514) - rollout_corr/kl:np.float64(0.001120272666678801) - rollout_corr/k3_kl:np.float64(0.000998899616035942) - rollout_corr/rollout_ppl:np.float64(1.3034690767526627) - rollout_corr/rollout_log_ppl:np.float64(0.25983976959832944) - rollout_corr/log_ppl_diff:np.float64(0.0011202737223356962) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024523677420802414) - rollout_corr/log_ppl_diff_max:np.float64(0.01079338788986206) - rollout_corr/log_ppl_diff_min:np.float64(-0.011696979403495789) - rollout_corr/ppl_ratio:np.float64(1.001125503797084) - rollout_corr/chi2_token:np.float64(0.0016837881412357092) - rollout_corr/chi2_seq:np.float64(0.4817654227372259) - actor/pg_loss:np.float64(0.0022594364127144217) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005095393812553084) - actor/ppo_kl:np.float64(0.000131376795138749) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(86914.96509032976) - self_distillation/token_reweight_w_std:np.float64(1369863.7220702441) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999393313191831) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041579058073693886) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3407774642109871) - perf/mfu/actor:np.float64(0.034170829781458016) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.09953689575195) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002358874771744013 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002358874771744013 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:287.7265625 - response_length/max:956.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.7265625 - response_length_non_aborted/max:956.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015220046043395996 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3281499948352575) - timing_s/agent_loop/generate_sequences/max:np.float64(6.804745152592659) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0793025807652157) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.804745152592659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:956 - timing_s/gen:12.672567792236805 - timing_s/reward:0.05679433047771454 - timing_s/old_log_prob:2.200064066797495 - timing_s/adv:0.5124469976872206 - timing_s/update_actor:15.322908757254481 - timing_s/step:30.852899638935924 - timing_s/stop_profile:0.00013393722474575043 - timing_per_token_ms/adv:0.0034909260438929423 - timing_per_token_ms/gen:0.17204604784594754 - timing_per_token_ms/update_actor:0.10438375381319728 - perf/total_num_tokens:146794 - perf/time_per_step:30.852899638935924 - perf/throughput:594.7334031723717 (TaskRunner pid=2026458) Training Progress: 76%|███████▌ | 76/100 [53:59<14:09, 35.40s/it] (TaskRunner pid=2026458) step:77 - global_seqlen/min:16068 - global_seqlen/max:21136 - global_seqlen/minmax_diff:5068 - global_seqlen/balanced_min:18347 - global_seqlen/balanced_max:18355 - global_seqlen/mean:18353.125 - actor/entropy:0.24358664453029633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3763711750507355 - training/rollout_probs_diff_mean:0.005085653159767389 - training/rollout_probs_diff_std:0.012961719185113907 - training/rollout_actor_probs_pearson_corr:0.9984139204025269 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999815821647644 - rollout_corr/rollout_is_ratio_fraction_high:1.5232524674502201e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.61626215535216e-05 - rollout_corr/rollout_is_max:2.443542003631592 - rollout_corr/rollout_is_min:0.0030977046117186546 - rollout_corr/rollout_is_std:0.04062869772315025 - rollout_corr/rollout_is_eff_sample_size:0.9983521480748362 - rollout_corr/rollout_is_seq_mean:1.0000320672988892 - rollout_corr/rollout_is_seq_std:0.0030800357926636934 - rollout_corr/rollout_is_seq_max:1.008837342262268 - rollout_corr/rollout_is_seq_min:0.9924508333206177 - rollout_corr/rollout_is_seq_max_deviation:0.008837342262268066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.30547040887177) - rollout_corr/training_log_ppl:np.float64(0.262247261358425) - rollout_corr/kl:np.float64(0.001232846368517393) - rollout_corr/k3_kl:np.float64(0.0012621798805412254) - rollout_corr/rollout_ppl:np.float64(1.303758448921144) - rollout_corr/rollout_log_ppl:np.float64(0.2610144141508499) - rollout_corr/log_ppl_diff:np.float64(0.0012328472075751051) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028479194588726386) - rollout_corr/log_ppl_diff_max:np.float64(0.01525259017944336) - rollout_corr/log_ppl_diff_min:np.float64(-0.007772982120513916) - rollout_corr/ppl_ratio:np.float64(1.0012400422710925) - rollout_corr/chi2_token:np.float64(0.0023893536999821663) - rollout_corr/chi2_seq:np.float64(0.8159184374380857) - actor/pg_loss:np.float64(0.002790838829241693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009325358823843999) - actor/ppo_kl:np.float64(0.0003359223030457992) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(180959.5167653456) - self_distillation/token_reweight_w_std:np.float64(2185127.907001757) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988750158809125) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06124208416440524) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4322587177157402) - perf/mfu/actor:np.float64(0.034241717008689325) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1767692565918) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021534981206059456 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021534981206059456 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:256.44140625 - response_length/max:782.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.44140625 - response_length_non_aborted/max:782.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013564899563789368 - timing_s/agent_loop/generate_sequences/min:np.float64(1.442766698077321) - timing_s/agent_loop/generate_sequences/max:np.float64(5.666013702750206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7960805580951273) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.666013702750206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:368 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:11.443953750655055 - timing_s/reward:0.054556334391236305 - timing_s/old_log_prob:2.218324951827526 - timing_s/adv:0.4792825896292925 - timing_s/update_actor:15.45035495236516 - timing_s/step:29.73319257237017 - timing_s/stop_profile:2.9521062970161438e-05 - timing_per_token_ms/adv:0.0032643118653450875 - timing_per_token_ms/gen:0.1743203057267446 - timing_per_token_ms/update_actor:0.10522972894510581 - perf/total_num_tokens:146825 - perf/time_per_step:29.73319257237017 - perf/throughput:617.2604894455499 (TaskRunner pid=2026458) Training Progress: 77%|███████▋ | 77/100 [54:28<12:55, 33.71s/it] (TaskRunner pid=2026458) step:78 - global_seqlen/min:15356 - global_seqlen/max:19696 - global_seqlen/minmax_diff:4340 - global_seqlen/balanced_min:17649 - global_seqlen/balanced_max:17655 - global_seqlen/mean:17652.75 - actor/entropy:0.23699037730693817 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975830912590027 - training/rollout_probs_diff_mean:0.005081570707261562 - training/rollout_probs_diff_std:0.013243689201772213 - training/rollout_actor_probs_pearson_corr:0.9983371496200562 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999217987060547 - rollout_corr/rollout_is_ratio_fraction_high:4.307003109715879e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.307003109715879e-05 - rollout_corr/rollout_is_max:2.9357402324676514 - rollout_corr/rollout_is_min:0.4458056390285492 - rollout_corr/rollout_is_std:0.04097927734255791 - rollout_corr/rollout_is_eff_sample_size:0.9983230388235971 - rollout_corr/rollout_is_seq_mean:1.0000085830688477 - rollout_corr/rollout_is_seq_std:0.002981689525768161 - rollout_corr/rollout_is_seq_max:1.0086852312088013 - rollout_corr/rollout_is_seq_min:0.9888802170753479 - rollout_corr/rollout_is_seq_max_deviation:0.0111197829246521 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3004110623151064) - rollout_corr/training_log_ppl:np.float64(0.257169630160206) - rollout_corr/kl:np.float64(0.0012380461494814199) - rollout_corr/k3_kl:np.float64(0.001083872083654569) - rollout_corr/rollout_ppl:np.float64(1.2987679266370833) - rollout_corr/rollout_log_ppl:np.float64(0.25593158273841254) - rollout_corr/log_ppl_diff:np.float64(0.0012380474217934534) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574279482359998) - rollout_corr/log_ppl_diff_max:np.float64(0.015157654881477356) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076891034841537476) - rollout_corr/ppl_ratio:np.float64(1.0012440220452845) - rollout_corr/chi2_token:np.float64(0.001788182882592082) - rollout_corr/chi2_seq:np.float64(1.9581220541149378) - actor/pg_loss:np.float64(0.0033810948953032494) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009038109546963824) - actor/ppo_kl:np.float64(0.00033419562218028886) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(189436.75045557343) - self_distillation/token_reweight_w_std:np.float64(2503367.794994272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009928282815963) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07160566429956816) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3780764937400818) - perf/mfu/actor:np.float64(0.03291894268049457) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0838165283203) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019830160308629274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019830160308629274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.0859375 - response_length/max:624.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.0859375 - response_length_non_aborted/max:624.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.438022971153259e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3432003632187843) - timing_s/agent_loop/generate_sequences/max:np.float64(5.082683224231005) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9419547133074957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.082683224231005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:624 - timing_s/gen:10.891948658972979 - timing_s/reward:0.054630380123853683 - timing_s/old_log_prob:2.192325107753277 - timing_s/adv:0.4803696293383837 - timing_s/update_actor:15.322288809344172 - timing_s/step:29.02985128760338 - timing_s/stop_profile:0.00014253705739974976 - timing_per_token_ms/adv:0.003401521217220997 - timing_per_token_ms/gen:0.15637219196274413 - timing_per_token_ms/update_actor:0.10849788849714756 - perf/total_num_tokens:141222 - perf/time_per_step:29.02985128760338 - perf/throughput:608.0895773495835 (TaskRunner pid=2026458) Training Progress: 78%|███████▊ | 78/100 [54:57<11:51, 32.32s/it] (TaskRunner pid=2026458) step:79 - global_seqlen/min:13318 - global_seqlen/max:19723 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16855 - global_seqlen/mean:16852.75 - actor/entropy:0.2380763292312622 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24131831526756287 - training/rollout_probs_diff_mean:0.00509747164323926 - training/rollout_probs_diff_std:0.01309633906930685 - training/rollout_actor_probs_pearson_corr:0.998324453830719 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.999850869178772 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4865467164781876e-05 - rollout_corr/rollout_is_max:1.6486144065856934 - rollout_corr/rollout_is_min:0.48596420884132385 - rollout_corr/rollout_is_std:0.04019506648182869 - rollout_corr/rollout_is_eff_sample_size:0.998386606117186 - rollout_corr/rollout_is_seq_mean:0.9999116659164429 - rollout_corr/rollout_is_seq_std:0.0028780419379472733 - rollout_corr/rollout_is_seq_max:1.0085704326629639 - rollout_corr/rollout_is_seq_min:0.9908472299575806 - rollout_corr/rollout_is_seq_max_deviation:0.009152770042419434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2909451411105692) - rollout_corr/training_log_ppl:np.float64(0.250693125941325) - rollout_corr/kl:np.float64(0.0009612457183649781) - rollout_corr/k3_kl:np.float64(0.0010165603632685816) - rollout_corr/rollout_ppl:np.float64(1.2896752119995654) - rollout_corr/rollout_log_ppl:np.float64(0.24973187799332663) - rollout_corr/log_ppl_diff:np.float64(0.0009612479479983449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024955651024356484) - rollout_corr/log_ppl_diff_max:np.float64(0.008482903242111206) - rollout_corr/log_ppl_diff_min:np.float64(-0.007829368114471436) - rollout_corr/ppl_ratio:np.float64(1.0009662464726716) - rollout_corr/chi2_token:np.float64(0.002142100827768445) - rollout_corr/chi2_seq:np.float64(0.36312326532788575) - actor/pg_loss:np.float64(0.0036202146438881755) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008493167902088317) - actor/ppo_kl:np.float64(1.813888876256442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(207000.82824485935) - self_distillation/token_reweight_w_std:np.float64(2774328.0958690653) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004315504338592) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07667552542989142) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5015818998217583) - perf/mfu/actor:np.float64(0.031159245026246143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.08302688598633) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000724691548384726 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000724691548384726 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:262.7734375 - response_length/max:630.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.7734375 - response_length_non_aborted/max:630.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013626180589199066 - timing_s/agent_loop/generate_sequences/min:np.float64(1.074292877689004) - timing_s/agent_loop/generate_sequences/max:np.float64(4.940646631643176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7851551758358255) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.940646631643176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:630 - timing_s/gen:11.244631016626954 - timing_s/reward:0.05328981205821037 - timing_s/old_log_prob:2.2739136423915625 - timing_s/adv:0.5032551754266024 - timing_s/update_actor:15.453622449189425 - timing_s/step:29.61371649429202 - timing_s/stop_profile:0.00012503564357757568 - timing_per_token_ms/adv:0.0037327377981828066 - timing_per_token_ms/gen:0.16715669714028472 - timing_per_token_ms/update_actor:0.1146224091705317 - perf/total_num_tokens:134822 - perf/time_per_step:29.61371649429202 - perf/throughput:569.0859505340484 (TaskRunner pid=2026458) Training Progress: 79%|███████▉ | 79/100 [55:27<11:02, 31.52s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but does not have the correct nitrile group and trifluoromethyl group structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" has a different structure with an oxygen atom and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a different structure with a double bond and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2026458) step:80 - global_seqlen/min:16122 - global_seqlen/max:23121 - global_seqlen/minmax_diff:6999 - global_seqlen/balanced_min:18851 - global_seqlen/balanced_max:18860 - global_seqlen/mean:18857.375 - actor/entropy:0.22489960491657257 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807961344718933 - training/rollout_probs_diff_mean:0.005050069652497768 - training/rollout_probs_diff_std:0.013410150073468685 - training/rollout_actor_probs_pearson_corr:0.998170018196106 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001280307769775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.252001963322982e-05 - rollout_corr/rollout_is_max:1.815222144126892 - rollout_corr/rollout_is_min:0.23840588331222534 - rollout_corr/rollout_is_std:0.040402136743068695 - rollout_corr/rollout_is_eff_sample_size:0.9983706838295904 - rollout_corr/rollout_is_seq_mean:1.0001842975616455 - rollout_corr/rollout_is_seq_std:0.002584717469289899 - rollout_corr/rollout_is_seq_max:1.0079708099365234 - rollout_corr/rollout_is_seq_min:0.9935450553894043 - rollout_corr/rollout_is_seq_max_deviation:0.007970809936523438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284651028458029) - rollout_corr/training_log_ppl:np.float64(0.2446684459282551) - rollout_corr/kl:np.float64(0.0011069629296169836) - rollout_corr/k3_kl:np.float64(0.0011388705046897485) - rollout_corr/rollout_ppl:np.float64(1.2831353461369872) - rollout_corr/rollout_log_ppl:np.float64(0.24356148090737406) - rollout_corr/log_ppl_diff:np.float64(0.0011069650208810344) - rollout_corr/log_ppl_abs_diff:np.float64(0.002547982890973799) - rollout_corr/log_ppl_diff_max:np.float64(0.011913180351257324) - rollout_corr/log_ppl_diff_min:np.float64(-0.00954301655292511) - rollout_corr/ppl_ratio:np.float64(1.0011127449106425) - rollout_corr/chi2_token:np.float64(0.0022547231055796146) - rollout_corr/chi2_seq:np.float64(0.705732943257317) - actor/pg_loss:np.float64(0.003671735990792513) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012357970999801182) - actor/ppo_kl:np.float64(0.00041334695177397407) - actor/pg_clipfrac_lower:np.float64(2.100134406646248e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(147860.0034218824) - self_distillation/token_reweight_w_std:np.float64(2243142.9639211292) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017643314786255) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08743216533912346) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.46084368228912354) - perf/mfu/actor:np.float64(0.0348035546181388) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0935287475586) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/score/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/score/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04910459662444814) - val-core/sciknoweval/acc/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07152449490492849) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8101809523809524) - val-core/sciknoweval/acc/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05480896953178682) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6956380952380953) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028665580321103334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0028665580321103334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.60546875 - response_length/max:780.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.60546875 - response_length_non_aborted/max:780.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012242421507835388 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5708349235355854) - timing_s/agent_loop/generate_sequences/max:np.float64(5.78690424002707) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9946807451196946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.78690424002707) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:780 - timing_s/gen:11.676643127575517 - timing_s/reward:0.060109492391347885 - timing_s/old_log_prob:2.263966953381896 - timing_s/adv:0.5860696248710155 - timing_s/update_actor:15.427983490750194 - timing_s/step:30.10305192694068 - timing_s/testing:71.47364217042923 - timing_s/save_checkpoint:6.949206165969372 - timing_s/stop_profile:0.00013513118028640747 - timing_per_token_ms/adv:0.0038848834002016158 - timing_per_token_ms/gen:0.16549703249345216 - timing_per_token_ms/update_actor:0.10226757098184526 - perf/total_num_tokens:150859 - perf/time_per_step:30.10305192694068 - perf/throughput:626.4273484883312 (TaskRunner pid=2026458) Training Progress: 80%|████████ | 80/100 [57:16<18:12, 54.65s/it] (TaskRunner pid=2026458) step:81 - global_seqlen/min:15749 - global_seqlen/max:23070 - global_seqlen/minmax_diff:7321 - global_seqlen/balanced_min:19345 - global_seqlen/balanced_max:19350 - global_seqlen/mean:19349.125 - actor/entropy:0.24268725514411926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.0049947695806622505 - training/rollout_probs_diff_std:0.013053881004452705 - training/rollout_actor_probs_pearson_corr:0.9983764290809631 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997670650482178 - rollout_corr/rollout_is_ratio_fraction_high:1.3156336535757873e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.262534614303149e-05 - rollout_corr/rollout_is_max:2.0260822772979736 - rollout_corr/rollout_is_min:0.2609400451183319 - rollout_corr/rollout_is_std:0.03953344747424126 - rollout_corr/rollout_is_eff_sample_size:0.9984388918274393 - rollout_corr/rollout_is_seq_mean:0.999757707118988 - rollout_corr/rollout_is_seq_std:0.002598333638161421 - rollout_corr/rollout_is_seq_max:1.0065761804580688 - rollout_corr/rollout_is_seq_min:0.9917704463005066 - rollout_corr/rollout_is_seq_max_deviation:0.008229553699493408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2969164904206991) - rollout_corr/training_log_ppl:np.float64(0.2555096454452723) - rollout_corr/kl:np.float64(0.0012044042446035519) - rollout_corr/k3_kl:np.float64(0.001068380127094315) - rollout_corr/rollout_ppl:np.float64(1.295284100342542) - rollout_corr/rollout_log_ppl:np.float64(0.25430524366674945) - rollout_corr/log_ppl_diff:np.float64(0.0012044017785228789) - rollout_corr/log_ppl_abs_diff:np.float64(0.002569538773968816) - rollout_corr/log_ppl_diff_max:np.float64(0.00988227128982544) - rollout_corr/log_ppl_diff_min:np.float64(-0.007966101169586182) - rollout_corr/ppl_ratio:np.float64(1.0012098925653845) - rollout_corr/chi2_token:np.float64(0.0018374514766037464) - rollout_corr/chi2_seq:np.float64(0.6602487270720303) - actor/pg_loss:np.float64(0.0036679430631920695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007375691257038852) - actor/ppo_kl:np.float64(0.00012186931077984298) - actor/pg_clipfrac_lower:np.float64(8.037551197048742e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(152346.46296780813) - self_distillation/token_reweight_w_std:np.float64(2223816.070475471) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006990339607) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07411914720432833) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.43313899636268616) - perf/mfu/actor:np.float64(0.035338759221170066) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.7124252319336) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0049204700626432896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0049204700626432896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.91015625 - response_length/max:761.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.91015625 - response_length_non_aborted/max:761.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.6648626923561096e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7210864126682281) - timing_s/agent_loop/generate_sequences/max:np.float64(5.891790520399809) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.159814590260794) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.891790520399809) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:761 - timing_s/gen:11.6525589954108 - timing_s/reward:0.05602879822254181 - timing_s/old_log_prob:2.4974898006767035 - timing_s/adv:0.5450900681316853 - timing_s/update_actor:15.660592023283243 - timing_s/step:30.51396298967302 - timing_s/stop_profile:0.00014164485037326813 - timing_per_token_ms/adv:0.0035214129071190897 - timing_per_token_ms/gen:0.15330499013815205 - timing_per_token_ms/update_actor:0.10117119006210386 - perf/total_num_tokens:154793 - perf/time_per_step:30.51396298967302 - perf/throughput:634.1072448225888 (TaskRunner pid=2026458) Training Progress: 81%|████████ | 81/100 [57:46<15:01, 47.43s/it] (TaskRunner pid=2026458) step:82 - global_seqlen/min:15004 - global_seqlen/max:21121 - global_seqlen/minmax_diff:6117 - global_seqlen/balanced_min:17893 - global_seqlen/balanced_max:17896 - global_seqlen/mean:17895.25 - actor/entropy:0.23795437812805176 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31782522797584534 - training/rollout_probs_diff_mean:0.004893430043011904 - training/rollout_probs_diff_std:0.012663507834076881 - training/rollout_actor_probs_pearson_corr:0.9984432458877563 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.000130534172058 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6648900529835373e-05 - rollout_corr/rollout_is_max:1.7479747533798218 - rollout_corr/rollout_is_min:0.3629961311817169 - rollout_corr/rollout_is_std:0.038970839232206345 - rollout_corr/rollout_is_eff_sample_size:0.998484170923625 - rollout_corr/rollout_is_seq_mean:1.0002964735031128 - rollout_corr/rollout_is_seq_std:0.002696148119866848 - rollout_corr/rollout_is_seq_max:1.0123590230941772 - rollout_corr/rollout_is_seq_min:0.9923802018165588 - rollout_corr/rollout_is_seq_max_deviation:0.012359023094177246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.296073100529611) - rollout_corr/training_log_ppl:np.float64(0.2552825820748694) - rollout_corr/kl:np.float64(0.0009574908228102696) - rollout_corr/k3_kl:np.float64(0.0011017269548574404) - rollout_corr/rollout_ppl:np.float64(1.2947798878885806) - rollout_corr/rollout_log_ppl:np.float64(0.254325088521) - rollout_corr/log_ppl_diff:np.float64(0.0009574935538694263) - rollout_corr/log_ppl_abs_diff:np.float64(0.002439796517137438) - rollout_corr/log_ppl_diff_max:np.float64(0.014934465289115906) - rollout_corr/log_ppl_diff_min:np.float64(-0.010826826095581055) - rollout_corr/ppl_ratio:np.float64(1.0009630019776523) - rollout_corr/chi2_token:np.float64(0.002452905522659421) - rollout_corr/chi2_seq:np.float64(0.423899469897151) - actor/pg_loss:np.float64(0.0041424884693697095) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013430351091301418) - actor/ppo_kl:np.float64(0.000410171516927349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(233069.21662305342) - self_distillation/token_reweight_w_std:np.float64(2612623.9466522) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009098192676902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0835862404492218) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.45514093339443207) - perf/mfu/actor:np.float64(0.03317730103506178) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.53085327148438) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003221185877919197 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003221185877919197 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.1640625 - response_length/max:790.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.1640625 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010550394654273987 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162438377737999) - timing_s/agent_loop/generate_sequences/max:np.float64(6.011627459898591) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.059660606115358) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.011627459898591) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:790 - timing_s/gen:12.346360467374325 - timing_s/reward:0.05675779469311237 - timing_s/old_log_prob:2.2316900305449963 - timing_s/adv:0.5058559346944094 - timing_s/update_actor:15.28704277612269 - timing_s/step:30.49560439772904 - timing_s/stop_profile:0.00011858716607093811 - timing_per_token_ms/adv:0.0035334511580895024 - timing_per_token_ms/gen:0.16450846725348867 - timing_per_token_ms/update_actor:0.10678142786579323 - perf/total_num_tokens:143162 - perf/time_per_step:30.49560439772904 - perf/throughput:586.8140787310525 (TaskRunner pid=2026458) Training Progress: 82%|████████▏ | 82/100 [58:17<12:42, 42.36s/it] (TaskRunner pid=2026458) step:83 - global_seqlen/min:15510 - global_seqlen/max:20545 - global_seqlen/minmax_diff:5035 - global_seqlen/balanced_min:18663 - global_seqlen/balanced_max:18665 - global_seqlen/mean:18664.0 - actor/entropy:0.253889262676239 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553972005844116 - training/rollout_probs_diff_mean:0.005415121093392372 - training/rollout_probs_diff_std:0.013402428478002548 - training/rollout_actor_probs_pearson_corr:0.9983406662940979 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.0002059936523438 - rollout_corr/rollout_is_ratio_fraction_high:2.7475547540234402e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3737773770117201e-05 - rollout_corr/rollout_is_max:2.8205859661102295 - rollout_corr/rollout_is_min:0.3914983570575714 - rollout_corr/rollout_is_std:0.04187990352511406 - rollout_corr/rollout_is_eff_sample_size:0.9982497385555893 - rollout_corr/rollout_is_seq_mean:1.000227689743042 - rollout_corr/rollout_is_seq_std:0.002809274010360241 - rollout_corr/rollout_is_seq_max:1.009268045425415 - rollout_corr/rollout_is_seq_min:0.9934084415435791 - rollout_corr/rollout_is_seq_max_deviation:0.009268045425415039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3143565426580608) - rollout_corr/training_log_ppl:np.float64(0.2695897765806876) - rollout_corr/kl:np.float64(0.000838180147326284) - rollout_corr/k3_kl:np.float64(0.001138548173230447) - rollout_corr/rollout_ppl:np.float64(1.313231692649424) - rollout_corr/rollout_log_ppl:np.float64(0.26875159548944794) - rollout_corr/log_ppl_diff:np.float64(0.0008381810912396759) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024882252619136125) - rollout_corr/log_ppl_diff_max:np.float64(0.013383328914642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.00783604383468628) - rollout_corr/ppl_ratio:np.float64(1.0008434464689344) - rollout_corr/chi2_token:np.float64(0.002809858415275812) - rollout_corr/chi2_seq:np.float64(0.9911151258274913) - actor/pg_loss:np.float64(0.004593900288455188) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947352435105131) - actor/ppo_kl:np.float64(0.00014004049228333315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(142169.04989654478) - self_distillation/token_reweight_w_std:np.float64(1783576.419247636) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992821563500911) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07421927264658734) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.4911562651395798) - perf/mfu/actor:np.float64(0.03439597980816678) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.20166397094727) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015317617217078805 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015317617217078805 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.34375 - response_length/max:727.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.34375 - response_length_non_aborted/max:727.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.794160723686218e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2851199880242348) - timing_s/agent_loop/generate_sequences/max:np.float64(5.475394682958722) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0284413718109136) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.475394682958722) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:727 - timing_s/gen:11.69941053725779 - timing_s/reward:0.055808091536164284 - timing_s/old_log_prob:2.32892463542521 - timing_s/adv:0.5197819676250219 - timing_s/update_actor:15.535809490829706 - timing_s/step:30.227494234219193 - timing_s/stop_profile:0.00012695416808128357 - timing_per_token_ms/adv:0.0034811801303647524 - timing_per_token_ms/gen:0.16072385066020703 - timing_per_token_ms/update_actor:0.10404930274076904 - perf/total_num_tokens:149312 - perf/time_per_step:30.227494234219193 - perf/throughput:617.4511143855027 (TaskRunner pid=2026458) Training Progress: 83%|████████▎ | 83/100 [58:47<10:58, 38.73s/it] (TaskRunner pid=2026458) step:84 - global_seqlen/min:14717 - global_seqlen/max:21368 - global_seqlen/minmax_diff:6651 - global_seqlen/balanced_min:17413 - global_seqlen/balanced_max:17422 - global_seqlen/mean:17419.375 - actor/entropy:0.22988727688789368 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35856932401657104 - training/rollout_probs_diff_mean:0.005215250421315432 - training/rollout_probs_diff_std:0.013718393631279469 - training/rollout_actor_probs_pearson_corr:0.9981431365013123 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9996470212936401 - rollout_corr/rollout_is_ratio_fraction_high:1.3454785403155256e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.072871423792094e-05 - rollout_corr/rollout_is_max:2.1147913932800293 - rollout_corr/rollout_is_min:0.390633225440979 - rollout_corr/rollout_is_std:0.041462238878011703 - rollout_corr/rollout_is_eff_sample_size:0.9982825263238067 - rollout_corr/rollout_is_seq_mean:0.9997439980506897 - rollout_corr/rollout_is_seq_std:0.0025515363086014986 - rollout_corr/rollout_is_seq_max:1.0085729360580444 - rollout_corr/rollout_is_seq_min:0.9917205572128296 - rollout_corr/rollout_is_seq_max_deviation:0.008572936058044434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2788517312146723) - rollout_corr/training_log_ppl:np.float64(0.24251381697831675) - rollout_corr/kl:np.float64(0.0013615037188472456) - rollout_corr/k3_kl:np.float64(0.0013387326044949077) - rollout_corr/rollout_ppl:np.float64(1.2770492630079389) - rollout_corr/rollout_log_ppl:np.float64(0.2411523106857203) - rollout_corr/log_ppl_diff:np.float64(0.0013615062925964594) - rollout_corr/log_ppl_abs_diff:np.float64(0.002768128179013729) - rollout_corr/log_ppl_diff_max:np.float64(0.011708617210388184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010206863284111023) - rollout_corr/ppl_ratio:np.float64(1.0013677747920156) - rollout_corr/chi2_token:np.float64(0.0024920080322772264) - rollout_corr/chi2_seq:np.float64(0.9803334202151746) - actor/pg_loss:np.float64(0.0052698974031955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015525902631452482) - actor/ppo_kl:np.float64(0.00019933821041862032) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(172811.0304318224) - self_distillation/token_reweight_w_std:np.float64(2611723.788451826) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009630725253373) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09219890262465924) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.5317341759800911) - perf/mfu/actor:np.float64(0.03233182941898891) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27649307250977) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011396203190088272 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011396203190088272 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.32421875 - response_length/max:900.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.32421875 - response_length_non_aborted/max:900.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003882218152284622 - timing_s/agent_loop/generate_sequences/min:np.float64(1.449787463992834) - timing_s/agent_loop/generate_sequences/max:np.float64(6.467833820730448) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1213262150922674) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.467833820730448) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:900 - timing_s/gen:12.418748388066888 - timing_s/reward:0.06932966038584709 - timing_s/old_log_prob:2.232461268082261 - timing_s/adv:0.5236291699111462 - timing_s/update_actor:15.485476253554225 - timing_s/step:30.817566089332104 - timing_s/stop_profile:0.000115908682346344 - timing_per_token_ms/adv:0.0037575197869552305 - timing_per_token_ms/gen:0.16709159194417458 - timing_per_token_ms/update_actor:0.1111225019091832 - perf/total_num_tokens:139355 - perf/time_per_step:30.817566089332104 - perf/throughput:565.2417504194123 (TaskRunner pid=2026458) Training Progress: 84%|████████▍ | 84/100 [59:18<09:42, 36.38s/it] (TaskRunner pid=2026458) step:85 - global_seqlen/min:16721 - global_seqlen/max:24217 - global_seqlen/minmax_diff:7496 - global_seqlen/balanced_min:19463 - global_seqlen/balanced_max:19465 - global_seqlen/mean:19464.25 - actor/entropy:0.23170782625675201 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5486323833465576 - training/rollout_probs_diff_mean:0.005258036311715841 - training/rollout_probs_diff_std:0.013953270390629768 - training/rollout_actor_probs_pearson_corr:0.998067319393158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0003063678741455 - rollout_corr/rollout_is_ratio_fraction_high:2.5328639821964316e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.065727964392863e-05 - rollout_corr/rollout_is_max:2.629549741744995 - rollout_corr/rollout_is_min:0.32894784212112427 - rollout_corr/rollout_is_std:0.04162293300032616 - rollout_corr/rollout_is_eff_sample_size:0.9982714780308726 - rollout_corr/rollout_is_seq_mean:1.0004109144210815 - rollout_corr/rollout_is_seq_std:0.002749430714175105 - rollout_corr/rollout_is_seq_max:1.0109564065933228 - rollout_corr/rollout_is_seq_min:0.9916076064109802 - rollout_corr/rollout_is_seq_max_deviation:0.010956406593322754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2771666692569852) - rollout_corr/training_log_ppl:np.float64(0.24135592477978207) - rollout_corr/kl:np.float64(0.0007578132510701607) - rollout_corr/k3_kl:np.float64(0.0010257670675173358) - rollout_corr/rollout_ppl:np.float64(1.2761707119643688) - rollout_corr/rollout_log_ppl:np.float64(0.24059811038023327) - rollout_corr/log_ppl_diff:np.float64(0.0007578143995488063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024299477954627946) - rollout_corr/log_ppl_diff_max:np.float64(0.011833712458610535) - rollout_corr/log_ppl_diff_min:np.float64(-0.010344773530960083) - rollout_corr/ppl_ratio:np.float64(1.0007629161700606) - rollout_corr/chi2_token:np.float64(0.0026269492227584124) - rollout_corr/chi2_seq:np.float64(1.3403879939578474) - actor/pg_loss:np.float64(0.002229055855423212) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019035061723116087) - actor/ppo_kl:np.float64(0.00024953764273760726) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(17317.446490210947) - self_distillation/token_reweight_w_std:np.float64(276465.0479241874) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991332795470953) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03143878703122027) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2347664088010788) - perf/mfu/actor:np.float64(0.03632172645099713) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27149963378906) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004859932523686439 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004859932523686439 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:308.4453125 - response_length/max:777.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.4453125 - response_length_non_aborted/max:777.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014561228454113007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4749425910413265) - timing_s/agent_loop/generate_sequences/max:np.float64(6.026074631139636) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2942245794984046) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.026074631139636) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:777 - timing_s/gen:12.095296228304505 - timing_s/reward:0.05645551718771458 - timing_s/old_log_prob:2.26230669952929 - timing_s/adv:0.4968310799449682 - timing_s/update_actor:15.393540350720286 - timing_s/step:30.395571315661073 - timing_s/stop_profile:0.00012073665857315063 - timing_per_token_ms/adv:0.0031906641660028527 - timing_per_token_ms/gen:0.15317869644011683 - timing_per_token_ms/update_actor:0.09885777997302932 - perf/total_num_tokens:155714 - perf/time_per_step:30.395571315661073 - perf/throughput:640.3646701640117 (TaskRunner pid=2026458) Training Progress: 85%|████████▌ | 85/100 [59:48<08:38, 34.60s/it] (TaskRunner pid=2026458) step:86 - global_seqlen/min:16265 - global_seqlen/max:20200 - global_seqlen/minmax_diff:3935 - global_seqlen/balanced_min:18647 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18649.75 - actor/entropy:0.23041389882564545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.898743212223053 - training/rollout_probs_diff_mean:0.005259583238512278 - training/rollout_probs_diff_std:0.014470859430730343 - training/rollout_actor_probs_pearson_corr:0.9979584813117981 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9998511075973511 - rollout_corr/rollout_is_ratio_fraction_high:3.836807809420861e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.394679803634062e-05 - rollout_corr/rollout_is_max:2.1972851753234863 - rollout_corr/rollout_is_min:0.0777815580368042 - rollout_corr/rollout_is_std:0.04242711886763573 - rollout_corr/rollout_is_eff_sample_size:0.9982026989977762 - rollout_corr/rollout_is_seq_mean:0.9998696446418762 - rollout_corr/rollout_is_seq_std:0.0025317175313830376 - rollout_corr/rollout_is_seq_max:1.009598731994629 - rollout_corr/rollout_is_seq_min:0.9924889802932739 - rollout_corr/rollout_is_seq_max_deviation:0.009598731994628906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2799476818181574) - rollout_corr/training_log_ppl:np.float64(0.24274212954333052) - rollout_corr/kl:np.float64(0.0012324780372026112) - rollout_corr/k3_kl:np.float64(0.0011905511769896293) - rollout_corr/rollout_ppl:np.float64(1.2783302562311292) - rollout_corr/rollout_log_ppl:np.float64(0.24150965179433115) - rollout_corr/log_ppl_diff:np.float64(0.0012324777489993721) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024851207563187927) - rollout_corr/log_ppl_diff_max:np.float64(0.009878866374492645) - rollout_corr/log_ppl_diff_min:np.float64(-0.00684240460395813) - rollout_corr/ppl_ratio:np.float64(1.0012374119833112) - rollout_corr/chi2_token:np.float64(0.0022098186891525984) - rollout_corr/chi2_seq:np.float64(0.6068784659728408) - actor/pg_loss:np.float64(0.0033142786705866456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009858607454589219) - actor/ppo_kl:np.float64(0.0001475137922213321) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(165894.66037644562) - self_distillation/token_reweight_w_std:np.float64(2346342.531521928) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011869312729686) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07381428830558434) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.42134569585323334) - perf/mfu/actor:np.float64(0.0348608857845462) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1626739501953) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0060413735918700695 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0060413735918700695 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.4296875 - response_length/max:825.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.4296875 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016151554882526398 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6213552672415972) - timing_s/agent_loop/generate_sequences/max:np.float64(6.171470927074552) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.300390396914736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.171470927074552) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.024075420573354 - timing_s/reward:0.05586143955588341 - timing_s/old_log_prob:2.3416189160197973 - timing_s/adv:0.5560324247926474 - timing_s/update_actor:15.314254153519869 - timing_s/step:30.382415380328894 - timing_s/stop_profile:0.0001287907361984253 - timing_per_token_ms/adv:0.0037268088365302976 - timing_per_token_ms/gen:0.15378022024009916 - timing_per_token_ms/update_actor:0.10264383003471808 - perf/total_num_tokens:149198 - perf/time_per_step:30.382415380328894 - perf/throughput:613.8336852597567 (TaskRunner pid=2026458) Training Progress: 86%|████████▌ | 86/100 [1:00:19<07:46, 33.35s/it] (TaskRunner pid=2026458) (TaskRunner pid=2026458) step:87 - global_seqlen/min:16648 - global_seqlen/max:20097 - global_seqlen/minmax_diff:3449 - global_seqlen/balanced_min:18341 - global_seqlen/balanced_max:18344 - global_seqlen/mean:18342.375 - actor/entropy:0.24005606770515442 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44461295008659363 - training/rollout_probs_diff_mean:0.005454737227410078 - training/rollout_probs_diff_std:0.014133700169622898 - training/rollout_actor_probs_pearson_corr:0.998093843460083 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9998968243598938 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.625958146993071e-05 - rollout_corr/rollout_is_max:1.91322922706604 - rollout_corr/rollout_is_min:0.3312951922416687 - rollout_corr/rollout_is_std:0.0426471121609211 - rollout_corr/rollout_is_eff_sample_size:0.998184050703313 - rollout_corr/rollout_is_seq_mean:0.999923050403595 - rollout_corr/rollout_is_seq_std:0.0029033836908638477 - rollout_corr/rollout_is_seq_max:1.0088640451431274 - rollout_corr/rollout_is_seq_min:0.990547776222229 - rollout_corr/rollout_is_seq_max_deviation:0.009452223777770996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2908267816528678) - rollout_corr/training_log_ppl:np.float64(0.25219142163405195) - rollout_corr/kl:np.float64(0.0013515868227500505) - rollout_corr/k3_kl:np.float64(0.001383622248795291) - rollout_corr/rollout_ppl:np.float64(1.2890665112063289) - rollout_corr/rollout_log_ppl:np.float64(0.25083983279182576) - rollout_corr/log_ppl_diff:np.float64(0.0013515888422261924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028635623457375914) - rollout_corr/log_ppl_diff_max:np.float64(0.012585937976837158) - rollout_corr/log_ppl_diff_min:np.float64(-0.009711101651191711) - rollout_corr/ppl_ratio:np.float64(1.0013586606364697) - rollout_corr/chi2_token:np.float64(0.0029732428956776857) - rollout_corr/chi2_seq:np.float64(0.6213931296952069) - actor/pg_loss:np.float64(0.005519538884982467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011970178129558917) - actor/ppo_kl:np.float64(0.00032003364054844496) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(104019.92419685167) - self_distillation/token_reweight_w_std:np.float64(1491533.3107083726) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000025857705623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07910635319421999) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.44405730813741684) - perf/mfu/actor:np.float64(0.03427082030528879) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2020378112793) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009922783821821213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009922783821821213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:277.73046875 - response_length/max:642.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:277.73046875 - response_length_non_aborted/max:642.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011511892080307007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3540627993643284) - timing_s/agent_loop/generate_sequences/max:np.float64(5.077815180644393) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9762204922371893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.077815180644393) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:642 - timing_s/gen:11.154959298670292 - timing_s/reward:0.057516856119036674 - timing_s/old_log_prob:2.265875956043601 - timing_s/adv:0.5384714491665363 - timing_s/update_actor:15.248491061851382 - timing_s/step:29.354633087292314 - timing_s/stop_profile:0.0001263357698917389 - timing_per_token_ms/adv:0.003669586470989555 - timing_per_token_ms/gen:0.15689333603384426 - timing_per_token_ms/update_actor:0.10391573516141846 - perf/total_num_tokens:146739 - perf/time_per_step:29.354633087292314 - perf/throughput:624.8545142926844 (TaskRunner pid=2026458) Training Progress: 87%|████████▋ | 87/100 [1:00:48<06:58, 32.18s/it] (TaskRunner pid=2026458) step:88 - global_seqlen/min:14981 - global_seqlen/max:20248 - global_seqlen/minmax_diff:5267 - global_seqlen/balanced_min:17171 - global_seqlen/balanced_max:17176 - global_seqlen/mean:17174.125 - actor/entropy:0.21586236357688904 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5279986262321472 - training/rollout_probs_diff_mean:0.005149980541318655 - training/rollout_probs_diff_std:0.014039847999811172 - training/rollout_actor_probs_pearson_corr:0.9979734420776367 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9998298287391663 - rollout_corr/rollout_is_ratio_fraction_high:2.9164297302486375e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4582148651243187e-05 - rollout_corr/rollout_is_max:2.4087512493133545 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.04135211929678917 - rollout_corr/rollout_is_eff_sample_size:0.998292149229813 - rollout_corr/rollout_is_seq_mean:0.9997440576553345 - rollout_corr/rollout_is_seq_std:0.0026299559976905584 - rollout_corr/rollout_is_seq_max:1.0093685388565063 - rollout_corr/rollout_is_seq_min:0.9928037524223328 - rollout_corr/rollout_is_seq_max_deviation:0.009368538856506348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.263124991208315) - rollout_corr/training_log_ppl:np.float64(0.23035154843819328) - rollout_corr/kl:np.float64(0.0012507079140435806) - rollout_corr/k3_kl:np.float64(0.0012822421521718752) - rollout_corr/rollout_ppl:np.float64(1.2614940297789872) - rollout_corr/rollout_log_ppl:np.float64(0.22910083959868643) - rollout_corr/log_ppl_diff:np.float64(0.0012507088395068422) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027966901106992736) - rollout_corr/log_ppl_diff_max:np.float64(0.014727145433425903) - rollout_corr/log_ppl_diff_min:np.float64(-0.014272093772888184) - rollout_corr/ppl_ratio:np.float64(1.0012579679023474) - rollout_corr/chi2_token:np.float64(0.002700346987694502) - rollout_corr/chi2_seq:np.float64(0.8957935450598598) - actor/pg_loss:np.float64(0.0045979529386386275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008685116490596556) - actor/ppo_kl:np.float64(0.00010619230280539682) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(126031.5825594815) - self_distillation/token_reweight_w_std:np.float64(1914441.9060592402) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010992304887623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07210871722782031) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4531179219484329) - perf/mfu/actor:np.float64(0.032191482757026275) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.15955352783203) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014600376598536968 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014600376598536968 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:267.87890625 - response_length/max:807.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:267.87890625 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012998655438423157 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3904380090534687) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8301611468195915) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8218457526090788) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8301611468195915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.686277892440557 - timing_s/reward:0.054150139912962914 - timing_s/old_log_prob:2.385304968804121 - timing_s/adv:0.4924223218113184 - timing_s/update_actor:15.133386766538024 - timing_s/step:29.843838276341558 - timing_s/stop_profile:5.164928734302521e-05 - timing_per_token_ms/adv:0.003584042286079483 - timing_per_token_ms/gen:0.170411040034422 - timing_per_token_ms/update_actor:0.11014670883187662 - perf/total_num_tokens:137393 - perf/time_per_step:29.843838276341558 - perf/throughput:575.466360626094 (TaskRunner pid=2026458) Training Progress: 88%|████████▊ | 88/100 [1:01:18<06:17, 31.49s/it] (TaskRunner pid=2026458) step:89 - global_seqlen/min:16152 - global_seqlen/max:22412 - global_seqlen/minmax_diff:6260 - global_seqlen/balanced_min:18887 - global_seqlen/balanced_max:18895 - global_seqlen/mean:18893.375 - actor/entropy:0.22386007010936737 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151239514350891 - training/rollout_probs_diff_mean:0.005173899233341217 - training/rollout_probs_diff_std:0.013607512228190899 - training/rollout_actor_probs_pearson_corr:0.9981110692024231 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:2.6122277631657198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.83668365329504e-05 - rollout_corr/rollout_is_max:2.51365327835083 - rollout_corr/rollout_is_min:0.1054598018527031 - rollout_corr/rollout_is_std:0.04179442301392555 - rollout_corr/rollout_is_eff_sample_size:0.9982565097660324 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0027215098962187767 - rollout_corr/rollout_is_seq_max:1.0083972215652466 - rollout_corr/rollout_is_seq_min:0.9915516376495361 - rollout_corr/rollout_is_seq_max_deviation:0.008448362350463867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2823212435469031) - rollout_corr/training_log_ppl:np.float64(0.24520724279864226) - rollout_corr/kl:np.float64(0.0008697826999508607) - rollout_corr/k3_kl:np.float64(0.0010809666277964425) - rollout_corr/rollout_ppl:np.float64(1.2811899655498564) - rollout_corr/rollout_log_ppl:np.float64(0.24433745855640154) - rollout_corr/log_ppl_diff:np.float64(0.000869784242240712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022608327853959054) - rollout_corr/log_ppl_diff_max:np.float64(0.010239839553833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071149468421936035) - rollout_corr/ppl_ratio:np.float64(1.0008741298224777) - rollout_corr/chi2_token:np.float64(0.0025997732300311327) - rollout_corr/chi2_seq:np.float64(0.5740627928171307) - actor/pg_loss:np.float64(0.0036603367188945413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008643267137813382) - actor/ppo_kl:np.float64(-9.36468737013918e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(95742.81214469112) - self_distillation/token_reweight_w_std:np.float64(1397763.4353546314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014505761209875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07612263548071496) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.43359871953725815) - perf/mfu/actor:np.float64(0.035258064824575705) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1541633605957) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00761464424431324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00761464424431324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.07421875 - response_length/max:1025.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.07421875 - response_length_non_aborted/max:1025.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2774325013160706e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3900669310241938) - timing_s/agent_loop/generate_sequences/max:np.float64(7.259778779000044) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.168158708969713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.259778779000044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1025 - timing_s/gen:12.976460268720984 - timing_s/reward:0.055463384836912155 - timing_s/old_log_prob:2.4733975417912006 - timing_s/adv:0.5145350992679596 - timing_s/update_actor:15.292766243219376 - timing_s/step:31.39907766878605 - timing_s/stop_profile:4.6625733375549316e-05 - timing_per_token_ms/adv:0.003404203188074918 - timing_per_token_ms/gen:0.16948735379649418 - timing_per_token_ms/update_actor:0.10117809975202535 - perf/total_num_tokens:151147 - perf/time_per_step:31.39907766878605 - perf/throughput:601.7175153772744 (TaskRunner pid=2026458) Training Progress: 89%|████████▉ | 89/100 [1:01:50<05:46, 31.47s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a chlorine atom, a trifluoromethyl group, and the correct aromatic ring structure. The formula is "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", which matches the required components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a chlorine atom but does not have the correct trifluoromethyl group or the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains all the necessary components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2026458) step:90 - global_seqlen/min:14894 - global_seqlen/max:19933 - global_seqlen/minmax_diff:5039 - global_seqlen/balanced_min:18191 - global_seqlen/balanced_max:18194 - global_seqlen/mean:18192.75 - actor/entropy:0.21710193157196045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3145221471786499 - training/rollout_probs_diff_mean:0.004972291644662619 - training/rollout_probs_diff_std:0.013361528515815735 - training/rollout_actor_probs_pearson_corr:0.9982044100761414 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0000026226043701 - rollout_corr/rollout_is_ratio_fraction_high:2.6879552024183795e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0319329855265096e-05 - rollout_corr/rollout_is_max:3.3088538646698 - rollout_corr/rollout_is_min:0.46167826652526855 - rollout_corr/rollout_is_std:0.041136063635349274 - rollout_corr/rollout_is_eff_sample_size:0.9983105639498434 - rollout_corr/rollout_is_seq_mean:1.0000332593917847 - rollout_corr/rollout_is_seq_std:0.0027625819202512503 - rollout_corr/rollout_is_seq_max:1.0086408853530884 - rollout_corr/rollout_is_seq_min:0.9904849529266357 - rollout_corr/rollout_is_seq_max_deviation:0.009515047073364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710709455423057) - rollout_corr/training_log_ppl:np.float64(0.23564479569904506) - rollout_corr/kl:np.float64(0.0014168473381213076) - rollout_corr/k3_kl:np.float64(0.0014046167735841664) - rollout_corr/rollout_ppl:np.float64(1.2692075530067086) - rollout_corr/rollout_log_ppl:np.float64(0.2342279483564198) - rollout_corr/log_ppl_diff:np.float64(0.0014168473426252604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028690413455478847) - rollout_corr/log_ppl_diff_max:np.float64(0.03278002142906189) - rollout_corr/log_ppl_diff_min:np.float64(-0.008600443601608276) - rollout_corr/ppl_ratio:np.float64(1.0014270655810833) - rollout_corr/chi2_token:np.float64(0.002751560416072607) - rollout_corr/chi2_seq:np.float64(0.7767476551234722) - actor/pg_loss:np.float64(0.0019601467065513134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009453206475882325) - actor/ppo_kl:np.float64(0.0005355499455532708) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(99385.55734024942) - self_distillation/token_reweight_w_std:np.float64(1449312.0253919773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012841576244682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045599023724207655) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.30510950461030006) - perf/mfu/actor:np.float64(0.03390671346645702) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.11214065551758) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/score/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/score/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/score/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05338104590757411) - val-core/sciknoweval/acc/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/acc/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07254164422675807) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8122666666666666) - val-core/sciknoweval/acc/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05053109084853488) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7015285714285714) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9995190476190476) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.001434898686445427) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997809523809524) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0009975480142912052) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9990666666666667) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0018903262505010432) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999142857142858) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006331005658902953) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981523809523809) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023204503433425093) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970095238095239) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023016113846578) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002840832807123661 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002840832807123661 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.6484375 - response_length/max:721.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.6484375 - response_length_non_aborted/max:721.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.825903594493866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7425885982811451) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7153176963329315) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0369765825307695) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7153176963329315) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:721 - timing_s/gen:11.199985072016716 - timing_s/reward:0.05536814592778683 - timing_s/old_log_prob:2.218005755916238 - timing_s/adv:0.4808800220489502 - timing_s/update_actor:15.297142412513494 - timing_s/step:29.337590511888266 - timing_s/testing:111.78063779324293 - timing_s/save_checkpoint:6.6935704834759235 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/adv:0.003304063583357039 - timing_per_token_ms/gen:0.15052529462700207 - timing_per_token_ms/update_actor:0.10510465990925984 - perf/total_num_tokens:145542 - perf/time_per_step:29.337590511888266 - perf/throughput:620.1173880529786 (TaskRunner pid=2026458) Training Progress: 90%|█████████ | 90/100 [1:04:17<11:03, 66.39s/it] (TaskRunner pid=2026458) step:91 - global_seqlen/min:15865 - global_seqlen/max:25244 - global_seqlen/minmax_diff:9379 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18818 - global_seqlen/mean:18815.125 - actor/entropy:0.2314702272415161 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21055352687835693 - training/rollout_probs_diff_mean:0.005051769316196442 - training/rollout_probs_diff_std:0.013311025686562061 - training/rollout_actor_probs_pearson_corr:0.9982390999794006 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999205470085144 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.47056513116695e-05 - rollout_corr/rollout_is_max:1.7587093114852905 - rollout_corr/rollout_is_min:0.23516227304935455 - rollout_corr/rollout_is_std:0.039755210280418396 - rollout_corr/rollout_is_eff_sample_size:0.9984218983709217 - rollout_corr/rollout_is_seq_mean:0.9999246001243591 - rollout_corr/rollout_is_seq_std:0.002499330323189497 - rollout_corr/rollout_is_seq_max:1.0098474025726318 - rollout_corr/rollout_is_seq_min:0.9917200207710266 - rollout_corr/rollout_is_seq_max_deviation:0.009847402572631836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2789378436282277) - rollout_corr/training_log_ppl:np.float64(0.24201013351557776) - rollout_corr/kl:np.float64(0.0009981181076650358) - rollout_corr/k3_kl:np.float64(0.0010161043154539584) - rollout_corr/rollout_ppl:np.float64(1.2776579768396914) - rollout_corr/rollout_log_ppl:np.float64(0.2410120142158121) - rollout_corr/log_ppl_diff:np.float64(0.0009981192997656763) - rollout_corr/log_ppl_abs_diff:np.float64(0.002134583191946149) - rollout_corr/log_ppl_diff_max:np.float64(0.00839272141456604) - rollout_corr/log_ppl_diff_min:np.float64(-0.010266929864883423) - rollout_corr/ppl_ratio:np.float64(1.0010020262561738) - rollout_corr/chi2_token:np.float64(0.0021203176584094763) - rollout_corr/chi2_seq:np.float64(0.651163290720433) - actor/pg_loss:np.float64(0.002993874717503786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000568103197565506) - actor/ppo_kl:np.float64(6.333214638765128e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(82354.48971123807) - self_distillation/token_reweight_w_std:np.float64(1252927.9291606762) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009026953484863) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06239277429995127) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.33179403841495514) - perf/mfu/actor:np.float64(0.034982824171721626) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.52531814575195) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002352050505578518 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002352050505578518 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.84765625 - response_length/max:674.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.84765625 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.271526217460632e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2008809298276901) - timing_s/agent_loop/generate_sequences/max:np.float64(5.367214092984796) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2526293304326828) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.367214092984796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.187385501340032 - timing_s/reward:0.05164775624871254 - timing_s/old_log_prob:2.4035099502652884 - timing_s/adv:0.5490099098533392 - timing_s/update_actor:15.261925529688597 - timing_s/step:29.56569225527346 - timing_s/stop_profile:0.00012163445353507996 - timing_per_token_ms/adv:0.0036473974385855742 - timing_per_token_ms/gen:0.14477741903821556 - timing_per_token_ms/update_actor:0.10139399505509927 - perf/total_num_tokens:150521 - perf/time_per_step:29.56569225527346 - perf/throughput:636.3837124985313 (TaskRunner pid=2026458) Training Progress: 91%|█████████ | 91/100 [1:04:47<08:18, 55.36s/it] (TaskRunner pid=2026458) step:92 - global_seqlen/min:16281 - global_seqlen/max:21475 - global_seqlen/minmax_diff:5194 - global_seqlen/balanced_min:18650 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18650.625 - actor/entropy:0.22619383037090302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.005189774092286825 - training/rollout_probs_diff_std:0.013963738456368446 - training/rollout_actor_probs_pearson_corr:0.9980257153511047 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999138116836548 - rollout_corr/rollout_is_ratio_fraction_high:1.3226638657215517e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011903974518645555 - rollout_corr/rollout_is_max:2.0864930152893066 - rollout_corr/rollout_is_min:0.0921701192855835 - rollout_corr/rollout_is_std:0.04230894520878792 - rollout_corr/rollout_is_eff_sample_size:0.9982129142824776 - rollout_corr/rollout_is_seq_mean:0.9999446272850037 - rollout_corr/rollout_is_seq_std:0.0027264179661870003 - rollout_corr/rollout_is_seq_max:1.0081113576889038 - rollout_corr/rollout_is_seq_min:0.9917219281196594 - rollout_corr/rollout_is_seq_max_deviation:0.008278071880340576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2724861456081271) - rollout_corr/training_log_ppl:np.float64(0.2372018521418795) - rollout_corr/kl:np.float64(0.0009739822945320498) - rollout_corr/k3_kl:np.float64(0.0010123744499423992) - rollout_corr/rollout_ppl:np.float64(1.2711997576989233) - rollout_corr/rollout_log_ppl:np.float64(0.23622786854684819) - rollout_corr/log_ppl_diff:np.float64(0.0009739835950313136) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024230604904005304) - rollout_corr/log_ppl_diff_max:np.float64(0.012229472398757935) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651081800460815) - rollout_corr/ppl_ratio:np.float64(1.0009791306219995) - rollout_corr/chi2_token:np.float64(0.0021006297320127487) - rollout_corr/chi2_seq:np.float64(0.7930810896214098) - actor/pg_loss:np.float64(0.002591841504909098) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004880208070972003) - actor/ppo_kl:np.float64(-2.9975729340492308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(72610.33176031034) - self_distillation/token_reweight_w_std:np.float64(1092734.639293266) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988934537395835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06326231267303228) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.3366621397435665) - perf/mfu/actor:np.float64(0.03433711138530011) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.60525131225586) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006365319713950157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006365319713950157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.33203125 - response_length/max:674.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.33203125 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.356330752372742e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4242578689008951) - timing_s/agent_loop/generate_sequences/max:np.float64(5.455314517021179) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2662257757328916) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.455314517021179) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:674 - timing_s/gen:11.507964111864567 - timing_s/reward:0.056888729333877563 - timing_s/old_log_prob:2.3034613393247128 - timing_s/adv:0.4925750717520714 - timing_s/update_actor:15.52449726127088 - timing_s/step:29.975734889507294 - timing_s/stop_profile:0.000132659450173378 - timing_per_token_ms/adv:0.0033013308652663875 - timing_per_token_ms/gen:0.15221168060134338 - timing_per_token_ms/update_actor:0.10404810335626072 - perf/total_num_tokens:149205 - perf/time_per_step:29.975734889507294 - perf/throughput:622.1907509106129 (TaskRunner pid=2026458) Training Progress: 92%|█████████▏| 92/100 [1:05:17<06:22, 47.76s/it] (TaskRunner pid=2026458) step:93 - global_seqlen/min:15058 - global_seqlen/max:22608 - global_seqlen/minmax_diff:7550 - global_seqlen/balanced_min:18025 - global_seqlen/balanced_max:18028 - global_seqlen/mean:18026.625 - actor/entropy:0.20888566970825195 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24053943157196045 - training/rollout_probs_diff_mean:0.004918403923511505 - training/rollout_probs_diff_std:0.013119952753186226 - training/rollout_actor_probs_pearson_corr:0.9981909394264221 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.98828125 - self_distillation/correct_sample_fraction:0.87890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9998151659965515 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263365528662689e-05 - rollout_corr/rollout_is_max:1.6370371580123901 - rollout_corr/rollout_is_min:0.2820625901222229 - rollout_corr/rollout_is_std:0.03976345434784889 - rollout_corr/rollout_is_eff_sample_size:0.9984207100389592 - rollout_corr/rollout_is_seq_mean:0.9998314380645752 - rollout_corr/rollout_is_seq_std:0.0023552256170660257 - rollout_corr/rollout_is_seq_max:1.0079485177993774 - rollout_corr/rollout_is_seq_min:0.9893273115158081 - rollout_corr/rollout_is_seq_max_deviation:0.010672688484191895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2420122399926186) - rollout_corr/training_log_ppl:np.float64(0.2130839267920237) - rollout_corr/kl:np.float64(0.0009293865148691793) - rollout_corr/k3_kl:np.float64(0.000889873810649533) - rollout_corr/rollout_ppl:np.float64(1.2408293401822448) - rollout_corr/rollout_log_ppl:np.float64(0.2121545401896583) - rollout_corr/log_ppl_diff:np.float64(0.0009293866023654118) - rollout_corr/log_ppl_abs_diff:np.float64(0.00208455762185622) - rollout_corr/log_ppl_diff_max:np.float64(0.008339852094650269) - rollout_corr/log_ppl_diff_min:np.float64(-0.005493566393852234) - rollout_corr/ppl_ratio:np.float64(1.0009328972082585) - rollout_corr/chi2_token:np.float64(0.00168294133618474) - rollout_corr/chi2_seq:np.float64(0.3678914944175631) - actor/pg_loss:np.float64(0.00139771425165236) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747084317888948) - actor/ppo_kl:np.float64(-7.94472919167788e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.87890625) - self_distillation/token_reweight_w_mean:np.float64(72035.54537807149) - self_distillation/token_reweight_w_std:np.float64(1075218.813784783) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001510763540864) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03214944276260212) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.2579457201063633) - perf/mfu/actor:np.float64(0.03364477703539987) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.39699172973633) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036366567946970463 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0036366567946970463 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:296.86328125 - response_length/max:640.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.86328125 - response_length_non_aborted/max:640.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017446838319301605 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6789815910160542) - timing_s/agent_loop/generate_sequences/max:np.float64(5.267441829666495) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.250441880845756) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.267441829666495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:640 - timing_s/gen:11.287975236773491 - timing_s/reward:0.0554285254329443 - timing_s/old_log_prob:2.2384933289140463 - timing_s/adv:0.5122805014252663 - timing_s/update_actor:15.286979779601097 - timing_s/step:29.47672649472952 - timing_s/stop_profile:0.00014898553490638733 - timing_per_token_ms/adv:0.003552249113639313 - timing_per_token_ms/gen:0.14853185305700872 - timing_per_token_ms/update_actor:0.106002786015138 - perf/total_num_tokens:144213 - perf/time_per_step:29.47672649472952 - perf/throughput:611.5545090538865 (TaskRunner pid=2026458) Training Progress: 93%|█████████▎| 93/100 [1:05:47<04:56, 42.29s/it] (TaskRunner pid=2026458) step:94 - global_seqlen/min:15786 - global_seqlen/max:20010 - global_seqlen/minmax_diff:4224 - global_seqlen/balanced_min:17735 - global_seqlen/balanced_max:17737 - global_seqlen/mean:17736.0 - actor/entropy:0.22735926508903503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311311483383179 - training/rollout_probs_diff_mean:0.005040409974753857 - training/rollout_probs_diff_std:0.013486634008586407 - training/rollout_actor_probs_pearson_corr:0.9981728792190552 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9998102784156799 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001036699177348055 - rollout_corr/rollout_is_max:1.7523070573806763 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03992203250527382 - rollout_corr/rollout_is_eff_sample_size:0.9984081138940653 - rollout_corr/rollout_is_seq_mean:0.9999222159385681 - rollout_corr/rollout_is_seq_std:0.002507504541426897 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9931005835533142 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2725239442661405) - rollout_corr/training_log_ppl:np.float64(0.23770636528206524) - rollout_corr/kl:np.float64(0.001132996877289294) - rollout_corr/k3_kl:np.float64(0.0012387144401486694) - rollout_corr/rollout_ppl:np.float64(1.2710481900721788) - rollout_corr/rollout_log_ppl:np.float64(0.23657336641917937) - rollout_corr/log_ppl_diff:np.float64(0.00113299886288587) - rollout_corr/log_ppl_abs_diff:np.float64(0.002540186673286371) - rollout_corr/log_ppl_diff_max:np.float64(0.02026122808456421) - rollout_corr/log_ppl_diff_min:np.float64(-0.015430659055709839) - rollout_corr/ppl_ratio:np.float64(1.0011393770109862) - rollout_corr/chi2_token:np.float64(0.002516603097319603) - rollout_corr/chi2_seq:np.float64(0.9786048375535756) - actor/pg_loss:np.float64(0.002404728322289884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001062956328951259) - actor/ppo_kl:np.float64(0.0001939034154609942) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(96479.46470297547) - self_distillation/token_reweight_w_std:np.float64(1265708.629157153) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990413470659405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.054301650350680575) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.35001390427351) - perf/mfu/actor:np.float64(0.032954674991088666) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4532012939453) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00508630508556962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00508630508556962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:301.4375 - response_length/max:631.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.4375 - response_length_non_aborted/max:631.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017067231237888336 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4371750131249428) - timing_s/agent_loop/generate_sequences/max:np.float64(5.477192688733339) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2628613221095293) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.477192688733339) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:631 - timing_s/gen:11.104732798412442 - timing_s/reward:0.05309521034359932 - timing_s/old_log_prob:2.3046134896576405 - timing_s/adv:0.4992757458239794 - timing_s/update_actor:15.339771771803498 - timing_s/step:29.391625313088298 - timing_s/stop_profile:0.000131336972117424 - timing_per_token_ms/adv:0.0035188017719890293 - timing_per_token_ms/gen:0.14390333815068995 - timing_per_token_ms/update_actor:0.10811183307822718 - perf/total_num_tokens:141888 - perf/time_per_step:29.391625313088298 - perf/throughput:603.4371971971906 (TaskRunner pid=2026458) Training Progress: 94%|█████████▍| 94/100 [1:06:16<03:50, 38.44s/it] (TaskRunner pid=2026458) step:95 - global_seqlen/min:14931 - global_seqlen/max:21336 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:17803 - global_seqlen/balanced_max:17812 - global_seqlen/mean:17810.75 - actor/entropy:0.22719933092594147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3546398878097534 - training/rollout_probs_diff_mean:0.004889350384473801 - training/rollout_probs_diff_std:0.012968857772648335 - training/rollout_actor_probs_pearson_corr:0.9983217716217041 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999182224273682 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3313806448422838e-05 - rollout_corr/rollout_is_max:1.646693468093872 - rollout_corr/rollout_is_min:0.012675890699028969 - rollout_corr/rollout_is_std:0.03914937749505043 - rollout_corr/rollout_is_eff_sample_size:0.9984694339690914 - rollout_corr/rollout_is_seq_mean:0.9998860955238342 - rollout_corr/rollout_is_seq_std:0.002519140485674143 - rollout_corr/rollout_is_seq_max:1.0078048706054688 - rollout_corr/rollout_is_seq_min:0.993129551410675 - rollout_corr/rollout_is_seq_max_deviation:0.00780487060546875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.269720992539078) - rollout_corr/training_log_ppl:np.float64(0.23537427955307066) - rollout_corr/kl:np.float64(0.000860856562256096) - rollout_corr/k3_kl:np.float64(0.0010757347620256041) - rollout_corr/rollout_ppl:np.float64(1.2686140895821154) - rollout_corr/rollout_log_ppl:np.float64(0.2345134206698276) - rollout_corr/log_ppl_diff:np.float64(0.0008608588832430542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024572820402681828) - rollout_corr/log_ppl_diff_max:np.float64(0.022655338048934937) - rollout_corr/log_ppl_diff_min:np.float64(-0.008849263191223145) - rollout_corr/ppl_ratio:np.float64(1.0008665048517287) - rollout_corr/chi2_token:np.float64(0.002582071116194129) - rollout_corr/chi2_seq:np.float64(1.1929384064860642) - actor/pg_loss:np.float64(0.0030269279377534986) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001186676310680923) - actor/ppo_kl:np.float64(-0.00011882417348374474) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(142327.77909193072) - self_distillation/token_reweight_w_std:np.float64(1811469.8086368854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001867031678557) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07137267180951312) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4086311012506485) - perf/mfu/actor:np.float64(0.03299815181672398) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.32865142822266) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007821861072443426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007821861072443426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.3984375 - response_length/max:742.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.3984375 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011094845831394196 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2119843941181898) - timing_s/agent_loop/generate_sequences/max:np.float64(5.851273892447352) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2298050526515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.851273892447352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:411 - timing_s/agent_loop/slowest/response_length:742 - timing_s/gen:11.692811168730259 - timing_s/reward:0.051967186853289604 - timing_s/old_log_prob:2.230885935947299 - timing_s/adv:0.5060996375977993 - timing_s/update_actor:15.389210676774383 - timing_s/step:29.97092897631228 - timing_s/stop_profile:0.0001416075974702835 - timing_per_token_ms/adv:0.003551925365283602 - timing_per_token_ms/gen:0.15567582437398828 - timing_per_token_ms/update_actor:0.10800507191425392 - perf/total_num_tokens:142486 - perf/time_per_step:29.97092897631228 - perf/throughput:594.2675321834984 (TaskRunner pid=2026458) Training Progress: 95%|█████████▌| 95/100 [1:06:46<02:59, 35.91s/it] (TaskRunner pid=2026458) step:96 - global_seqlen/min:15504 - global_seqlen/max:21947 - global_seqlen/minmax_diff:6443 - global_seqlen/balanced_min:17847 - global_seqlen/balanced_max:17850 - global_seqlen/mean:17848.25 - actor/entropy:0.24302901327610016 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7704145312309265 - training/rollout_probs_diff_mean:0.0050596678629517555 - training/rollout_probs_diff_std:0.013549817726016045 - training/rollout_actor_probs_pearson_corr:0.9982749223709106 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000941753387451 - rollout_corr/rollout_is_ratio_fraction_high:4.025656744488515e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.709428271278739e-05 - rollout_corr/rollout_is_max:2.5671136379241943 - rollout_corr/rollout_is_min:0.13419120013713837 - rollout_corr/rollout_is_std:0.039693690836429596 - rollout_corr/rollout_is_eff_sample_size:0.9984272458997584 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0026221859734505415 - rollout_corr/rollout_is_seq_max:1.009491205215454 - rollout_corr/rollout_is_seq_min:0.9908090233802795 - rollout_corr/rollout_is_seq_max_deviation:0.009491205215454102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2904391190968454) - rollout_corr/training_log_ppl:np.float64(0.2496467249293346) - rollout_corr/kl:np.float64(0.0008864650635374005) - rollout_corr/k3_kl:np.float64(0.0009545528546368587) - rollout_corr/rollout_ppl:np.float64(1.2892436645925045) - rollout_corr/rollout_log_ppl:np.float64(0.24876026091806125) - rollout_corr/log_ppl_diff:np.float64(0.0008864640112733468) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021889270428800955) - rollout_corr/log_ppl_diff_max:np.float64(0.013020426034927368) - rollout_corr/log_ppl_diff_min:np.float64(-0.006367728114128113) - rollout_corr/ppl_ratio:np.float64(1.0008906812872738) - rollout_corr/chi2_token:np.float64(0.0020458281505852938) - rollout_corr/chi2_seq:np.float64(0.6757269599474967) - actor/pg_loss:np.float64(0.0028436718275770545) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006703252311126562) - actor/ppo_kl:np.float64(0.0001225946314420412) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(73408.10212582699) - self_distillation/token_reweight_w_std:np.float64(1221313.2646529041) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004472548607737) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07176223069836851) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.3494473174214363) - perf/mfu/actor:np.float64(0.03336222886368918) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.36438751220703) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025512599386274815 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025512599386274815 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1015625 - response_length/max:636.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1015625 - response_length_non_aborted/max:636.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012776069343090057 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3372939508408308) - timing_s/agent_loop/generate_sequences/max:np.float64(5.064935522153974) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.084805842998321) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.064935522153974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:572 - timing_s/gen:11.018641920760274 - timing_s/reward:0.05196058563888073 - timing_s/old_log_prob:2.2254901994019747 - timing_s/adv:0.4944445975124836 - timing_s/update_actor:15.122869728133082 - timing_s/step:29.00846175290644 - timing_s/stop_profile:2.9573217034339905e-05 - timing_per_token_ms/adv:0.0034628366752516605 - timing_per_token_ms/gen:0.14785757119723403 - timing_per_token_ms/update_actor:0.10591283268761001 - perf/total_num_tokens:142786 - perf/time_per_step:29.00846175290644 - perf/throughput:615.2773681014552 (TaskRunner pid=2026458) Training Progress: 96%|█████████▌| 96/100 [1:07:15<02:15, 33.85s/it] (TaskRunner pid=2026458) step:97 - global_seqlen/min:16071 - global_seqlen/max:23461 - global_seqlen/minmax_diff:7390 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18815 - global_seqlen/mean:18813.875 - actor/entropy:0.2445484697818756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4984980821609497 - training/rollout_probs_diff_mean:0.005207015201449394 - training/rollout_probs_diff_std:0.013489038683474064 - training/rollout_actor_probs_pearson_corr:0.9982696175575256 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.9999282360076904 - rollout_corr/rollout_is_ratio_fraction_high:1.330795930698514e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.65397965349257e-05 - rollout_corr/rollout_is_max:3.3646888732910156 - rollout_corr/rollout_is_min:0.3885670304298401 - rollout_corr/rollout_is_std:0.04048614203929901 - rollout_corr/rollout_is_eff_sample_size:0.9983633169728505 - rollout_corr/rollout_is_seq_mean:0.9999846816062927 - rollout_corr/rollout_is_seq_std:0.002680700272321701 - rollout_corr/rollout_is_seq_max:1.0078043937683105 - rollout_corr/rollout_is_seq_min:0.9888970255851746 - rollout_corr/rollout_is_seq_max_deviation:0.01110297441482544 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3025756301358342) - rollout_corr/training_log_ppl:np.float64(0.2600360629003262) - rollout_corr/kl:np.float64(0.001345034299074399) - rollout_corr/k3_kl:np.float64(0.0010471967440253138) - rollout_corr/rollout_ppl:np.float64(1.3007347490638494) - rollout_corr/rollout_log_ppl:np.float64(0.2586910269019427) - rollout_corr/log_ppl_diff:np.float64(0.001345035998383537) - rollout_corr/log_ppl_abs_diff:np.float64(0.002600655221613124) - rollout_corr/log_ppl_diff_max:np.float64(0.013873368501663208) - rollout_corr/log_ppl_diff_min:np.float64(-0.008637338876724243) - rollout_corr/ppl_ratio:np.float64(1.0013508247211576) - rollout_corr/chi2_token:np.float64(0.001424239482730627) - rollout_corr/chi2_seq:np.float64(0.4958528857678175) - actor/pg_loss:np.float64(0.0023965955479070544) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000528412345374818) - actor/ppo_kl:np.float64(0.00046769405014401855) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(76633.93915633648) - self_distillation/token_reweight_w_std:np.float64(1211350.6270655608) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995253349188715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0652933964447584) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.31651656329631805) - perf/mfu/actor:np.float64(0.03500956607334557) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4477882385254) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004978840239346027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004978840239346027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.52734375 - response_length/max:730.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.52734375 - response_length_non_aborted/max:730.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.985447645187378e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1220364272594452) - timing_s/agent_loop/generate_sequences/max:np.float64(5.692030295729637) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1152001094887964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.692030295729637) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:730 - timing_s/gen:11.270145887508988 - timing_s/reward:0.05703514441847801 - timing_s/old_log_prob:2.3115832544863224 - timing_s/adv:0.5203931517899036 - timing_s/update_actor:15.463589219376445 - timing_s/step:29.711128624156117 - timing_s/stop_profile:0.00012691877782344818 - timing_per_token_ms/adv:0.0034575090976068437 - timing_per_token_ms/gen:0.14998264492379848 - timing_per_token_ms/update_actor:0.10274059184628662 - perf/total_num_tokens:150511 - perf/time_per_step:29.711128624156117 - perf/throughput:633.2265340032793 (TaskRunner pid=2026458) Training Progress: 97%|█████████▋| 97/100 [1:07:45<01:37, 32.62s/it] (TaskRunner pid=2026458) step:98 - global_seqlen/min:14853 - global_seqlen/max:23186 - global_seqlen/minmax_diff:8333 - global_seqlen/balanced_min:19016 - global_seqlen/balanced_max:19021 - global_seqlen/mean:19019.875 - actor/entropy:0.23993536829948425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2247922420501709 - training/rollout_probs_diff_mean:0.005032053682953119 - training/rollout_probs_diff_std:0.013124685734510422 - training/rollout_actor_probs_pearson_corr:0.99836266040802 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9997885227203369 - rollout_corr/rollout_is_ratio_fraction_high:2.6424617317388766e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.9636925066588446e-05 - rollout_corr/rollout_is_max:2.206674814224243 - rollout_corr/rollout_is_min:0.19827817380428314 - rollout_corr/rollout_is_std:0.04013793542981148 - rollout_corr/rollout_is_eff_sample_size:0.9983908838332385 - rollout_corr/rollout_is_seq_mean:0.9997617602348328 - rollout_corr/rollout_is_seq_std:0.0025118952617049217 - rollout_corr/rollout_is_seq_max:1.0087740421295166 - rollout_corr/rollout_is_seq_min:0.9924179315567017 - rollout_corr/rollout_is_seq_max_deviation:0.008774042129516602 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2941957069560885) - rollout_corr/training_log_ppl:np.float64(0.254570292920107) - rollout_corr/kl:np.float64(0.0012180306670934726) - rollout_corr/k3_kl:np.float64(0.000970762781207668) - rollout_corr/rollout_ppl:np.float64(1.2925921510905027) - rollout_corr/rollout_log_ppl:np.float64(0.25335226202150807) - rollout_corr/log_ppl_diff:np.float64(0.001218030898598954) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023932511976454407) - rollout_corr/log_ppl_diff_max:np.float64(0.008434802293777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.005523428320884705) - rollout_corr/ppl_ratio:np.float64(1.0012223366647959) - rollout_corr/chi2_token:np.float64(0.0014376682229340076) - rollout_corr/chi2_seq:np.float64(0.6267413701862097) - actor/pg_loss:np.float64(0.003052193787880242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006503687072836328) - actor/ppo_kl:np.float64(0.000135291372203028) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(107416.14729232923) - self_distillation/token_reweight_w_std:np.float64(1626133.0662139303) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999873915920034) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0583975906483829) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.36374419182538986) - perf/mfu/actor:np.float64(0.03551806757350935) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40508651733398) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001578870927914977 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001578870927914977 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.65234375 - response_length/max:718.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.65234375 - response_length_non_aborted/max:718.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011442229151725769 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4412503708153963) - timing_s/agent_loop/generate_sequences/max:np.float64(5.591642700135708) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1361403591217822) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.591642700135708) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:718 - timing_s/gen:11.411643352359533 - timing_s/reward:0.05464274063706398 - timing_s/old_log_prob:2.2537942565977573 - timing_s/adv:0.5091272983700037 - timing_s/update_actor:15.217187775298953 - timing_s/step:29.54008462652564 - timing_s/stop_profile:0.00011652149260044098 - timing_per_token_ms/adv:0.003346021585118223 - timing_per_token_ms/gen:0.1507741534525022 - timing_per_token_ms/update_actor:0.1000084633528017 - perf/total_num_tokens:152159 - perf/time_per_step:29.54008462652564 - perf/throughput:643.8666388559031 (TaskRunner pid=2026458) Training Progress: 98%|█████████▊| 98/100 [1:08:14<01:03, 31.71s/it] (TaskRunner pid=2026458) step:99 - global_seqlen/min:16270 - global_seqlen/max:23777 - global_seqlen/minmax_diff:7507 - global_seqlen/balanced_min:18894 - global_seqlen/balanced_max:18898 - global_seqlen/mean:18896.125 - actor/entropy:0.23954525589942932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5315115451812744 - training/rollout_probs_diff_mean:0.005148096010088921 - training/rollout_probs_diff_std:0.013385861180722713 - training/rollout_actor_probs_pearson_corr:0.9983036518096924 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.00002920627594 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.40447464218596e-05 - rollout_corr/rollout_is_max:1.9394363164901733 - rollout_corr/rollout_is_min:0.2951362729072571 - rollout_corr/rollout_is_std:0.04090065881609917 - rollout_corr/rollout_is_eff_sample_size:0.9983299298399158 - rollout_corr/rollout_is_seq_mean:1.0000901222229004 - rollout_corr/rollout_is_seq_std:0.0026464853435754776 - rollout_corr/rollout_is_seq_max:1.00861656665802 - rollout_corr/rollout_is_seq_min:0.9930309057235718 - rollout_corr/rollout_is_seq_max_deviation:0.00861656665802002 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2940249065868556) - rollout_corr/training_log_ppl:np.float64(0.25375668989727274) - rollout_corr/kl:np.float64(0.0008266884788987738) - rollout_corr/k3_kl:np.float64(0.0009614558624662095) - rollout_corr/rollout_ppl:np.float64(1.2929239594377577) - rollout_corr/rollout_log_ppl:np.float64(0.2529299992165761) - rollout_corr/log_ppl_diff:np.float64(0.000826690680696629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025123875675490126) - rollout_corr/log_ppl_diff_max:np.float64(0.009278476238250732) - rollout_corr/log_ppl_diff_min:np.float64(-0.007872521877288818) - rollout_corr/ppl_ratio:np.float64(1.000831762328744) - rollout_corr/chi2_token:np.float64(0.002216932363808155) - rollout_corr/chi2_seq:np.float64(0.6580137938726693) - actor/pg_loss:np.float64(0.004063432686962187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007904909580247477) - actor/ppo_kl:np.float64(4.468652047506794e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(131557.28193879593) - self_distillation/token_reweight_w_std:np.float64(2127779.07550747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999837173614651) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07958601297286805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4794868528842926) - perf/mfu/actor:np.float64(0.035181320287522105) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.34246444702148) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003239069599658251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003239069599658251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.12890625 - response_length/max:681.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:278.12890625 - response_length_non_aborted/max:681.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017566047608852386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2292504087090492) - timing_s/agent_loop/generate_sequences/max:np.float64(5.23788345977664) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0256088213573094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.23788345977664) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:681 - timing_s/gen:11.295774277299643 - timing_s/reward:0.05612022429704666 - timing_s/old_log_prob:2.242926351726055 - timing_s/adv:0.5197359751909971 - timing_s/update_actor:15.403783183544874 - timing_s/step:29.607021318748593 - timing_s/stop_profile:0.00011566095054149628 - timing_per_token_ms/adv:0.0034381121472722392 - timing_per_token_ms/gen:0.1586462869524254 - timing_per_token_ms/update_actor:0.101897764644503 - perf/total_num_tokens:151169 - perf/time_per_step:29.607021318748593 - perf/throughput:638.2312086232755 (TaskRunner pid=2026458) Training Progress: 99%|█████████▉| 99/100 [1:08:44<00:31, 31.09s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains the correct structure with a chlorine atom, a trifluoromethyl group, and the necessary aromatic ring, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but lacks the correct trifluoromethyl group and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" contains a trifluoromethyl group and a chlorine atom but has an incorrect aromatic ring structure and additional functional groups not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the only option that matches the product structure is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2026458) step:100 - global_seqlen/min:17070 - global_seqlen/max:22010 - global_seqlen/minmax_diff:4940 - global_seqlen/balanced_min:19621 - global_seqlen/balanced_max:19622 - global_seqlen/mean:19621.75 - actor/entropy:0.21838146448135376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163663625717163 - training/rollout_probs_diff_mean:0.004920980893075466 - training/rollout_probs_diff_std:0.013346283696591854 - training/rollout_actor_probs_pearson_corr:0.9981935024261475 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000070333480835 - rollout_corr/rollout_is_ratio_fraction_high:1.2321644135226961e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.16082179476507e-05 - rollout_corr/rollout_is_max:2.1037662029266357 - rollout_corr/rollout_is_min:0.39213550090789795 - rollout_corr/rollout_is_std:0.04028838127851486 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:1.0000463724136353 - rollout_corr/rollout_is_seq_std:0.002407139865681529 - rollout_corr/rollout_is_seq_max:1.0090001821517944 - rollout_corr/rollout_is_seq_min:0.9934689998626709 - rollout_corr/rollout_is_seq_max_deviation:0.009000182151794434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2685748748481274) - rollout_corr/training_log_ppl:np.float64(0.23310650100756902) - rollout_corr/kl:np.float64(0.0009287315268036167) - rollout_corr/k3_kl:np.float64(0.001084987000297133) - rollout_corr/rollout_ppl:np.float64(1.2673596912063658) - rollout_corr/rollout_log_ppl:np.float64(0.2321777678298531) - rollout_corr/log_ppl_diff:np.float64(0.0009287331777159125) - rollout_corr/log_ppl_abs_diff:np.float64(0.002280793822137639) - rollout_corr/log_ppl_diff_max:np.float64(0.015468984842300415) - rollout_corr/log_ppl_diff_min:np.float64(-0.010868668556213379) - rollout_corr/ppl_ratio:np.float64(1.0009334608912468) - rollout_corr/chi2_token:np.float64(0.002497109118849039) - rollout_corr/chi2_seq:np.float64(1.0585334873758256) - actor/pg_loss:np.float64(0.0023272527614608407) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004233177678543143) - actor/ppo_kl:np.float64(0.00010989303795483352) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(73094.00544022233) - self_distillation/token_reweight_w_std:np.float64(1188161.9467263469) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000056762015447) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03758277394808829) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.25844827108085155) - perf/mfu/actor:np.float64(0.03632957727570209) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.29161834716797) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/reward/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/score/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/score/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03834834240388913) - val-core/sciknoweval/acc/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/acc/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.05779624954713381) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8013714285714286) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03927314998986781) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7074904761904762) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994476190476191) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015248805759714617) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011569664768572288) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988761904761905) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0020220104047945714) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999476190476191) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0004966783087710074) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980238095238095) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.002346295386522406) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9967095238095238) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0022003864842959084) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011474530911073089 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011474530911073089 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:317.0234375 - response_length/max:616.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.0234375 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018987245857715607 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5517499037086964) - timing_s/agent_loop/generate_sequences/max:np.float64(5.5460103284567595) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.556732240387646) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.5460103284567595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:602 - timing_s/gen:11.436559945344925 - timing_s/reward:0.05615238472819328 - timing_s/old_log_prob:2.225490480661392 - timing_s/adv:0.4910304043442011 - timing_s/update_actor:15.488128297030926 - timing_s/step:29.756166722625494 - timing_s/testing:108.75369282439351 - timing_s/save_checkpoint:7.179236024618149 - timing_s/stop_profile:0.00012516416609287262 - timing_per_token_ms/adv:0.003128100222611395 - timing_per_token_ms/gen:0.14091722252082264 - timing_per_token_ms/update_actor:0.0986668384384097 - perf/total_num_tokens:156974 - perf/time_per_step:29.756166722625494 - perf/throughput:659.4179345379304 (TaskRunner pid=2026458) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2026458) 'np.float64(0.03834834240388913), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2026458) 'np.float64(0.999702380952381), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2026458) 'np.float64(0.0011526736149426837), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999904761904761), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0002127457895589398), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9994476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0015248805759714617), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9997), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0011569664768572288), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9988761904761905), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0020220104047945714), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0004966783087710074), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2026458) 'np.float64(0.9980238095238095), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2026458) 'np.float64(0.002346295386522406), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2026458) 'np.float64(0.9967095238095238), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2026458) 'np.float64(0.0022003864842959084), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2026458) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2026458) Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 65.49s/it] Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 42.70s/it] (TaskRunner pid=2026458) wandb: updating run metadata (TaskRunner pid=2026458) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2026458) wandb: uploading output.log (TaskRunner pid=2026458) wandb: uploading history steps 98-99, summary, console lines 651-774 (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run history: (TaskRunner pid=2026458) wandb: actor/entropy ▂▁▂▅▄█▅▅▃▆▆▆▇█▆▅▆▆▄▆▃▂▄▅▄▃▃▄▃▄▃▃▄▂▂▂▂▂▃▂ (TaskRunner pid=2026458) wandb: actor/grad_norm ▆▄▆▆▆▄▆▅▄▅▆▇▅▄▄▄▃▄▃▃▄▃█▃▄▄▃▁▂▃▃▂▂▃▃▁▃▃▂▂ (TaskRunner pid=2026458) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/lr ▁▃▄▅▅▇██████████████████████████████████ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac ▁▅▆▆▅▅▄▅▃█▃▆▄▄▆▃▃▆▃▃▃▅▃▇▆▄▃▄▄▄▅▃▃▄▅▂▄▂▅▂ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▅▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/pg_loss █▅▅▆▆▄▇▅▄▄▃▃▃▄▁▂▃▂▄▃▃▃▃▂▃▂▃▂▂▂▃▃▃▄▄▁▂▂▂▂ (TaskRunner pid=2026458) wandb: actor/ppo_kl ▃▄▁▃▆▂▅▃▄▂▁▃▃█▆▆▄▄▅▅▃▅▆▂▆▆▆▂█▄▃▄▆▄▅▆█▄▇▄ (TaskRunner pid=2026458) wandb: critic/advantages/max █████████▃██▆████████████████▆██▆▆▆▁████ (TaskRunner pid=2026458) wandb: critic/advantages/mean ▃█▅▄▃▂▃▄▁▂▃▄▄▁▄▃▄▂▃▅▄▄▃▄▃▂▄▃▃▄▄▂▄▁▄▃▃▄▂▃ (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run summary: (TaskRunner pid=2026458) wandb: actor/entropy 0.21838 (TaskRunner pid=2026458) wandb: actor/grad_norm 0.25845 (TaskRunner pid=2026458) wandb: actor/kl_loss 0 (TaskRunner pid=2026458) wandb: actor/lr 0.0 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac 0.00042 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2026458) wandb: actor/pg_loss 0.00233 (TaskRunner pid=2026458) wandb: actor/ppo_kl 0.00011 (TaskRunner pid=2026458) wandb: critic/advantages/max 0.75 (TaskRunner pid=2026458) wandb: critic/advantages/mean -0.00115 (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: 🚀 View run qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/bz6p2yxy (TaskRunner pid=2026458) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2026458) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2026458) wandb: Find logs at: ./wandb/run-20260702_062054-bz6p2yxy/logs (TaskRunner pid=2026458) Exception ignored in atexit callback: .teardown_atexit at 0x7a92e933fb00> (TaskRunner pid=2026458) Traceback (most recent call last): (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2026458) conn.teardown(hooks.exit_code) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2026458) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2026458) return future.result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2026458) return self.__get_result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2026458) raise self._exception (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2026458) return await fn() (TaskRunner pid=2026458) ^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2026458) await self._client.close() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2026458) await self._writer.wait_closed() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2026458) await self._protocol._get_close_waiter(self) (TaskRunner pid=2026458) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 07:32:12] Finished chemistry rlsd_tr [2026-07-02 07:32:12] Starting physics grpo Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_4B 2026-07-02 07:32:14,409 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 07:32:24,172 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2049544) TaskRunner hostname: mole-workspace-rw, PID: 2049544 (TaskRunner pid=2049544) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'calculate_entropy': False, (TaskRunner pid=2049544) 'calculate_sum_pi_squared': False, (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra']}, (TaskRunner pid=2049544) 'clip_ratio': 0.2, (TaskRunner pid=2049544) 'clip_ratio_c': 3.0, (TaskRunner pid=2049544) 'clip_ratio_high': 0.28, (TaskRunner pid=2049544) 'clip_ratio_low': 0.2, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_coeff': 0, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'freeze_vision_tower': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'kl_loss_coef': 0.0, (TaskRunner pid=2049544) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'loss_scale_factor': None, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-06, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': 10, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2049544) 'clip_cov_lb': 1.0, (TaskRunner pid=2049544) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'clip_cov_ub': 5.0, (TaskRunner pid=2049544) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'loss_mode': 'vanilla', (TaskRunner pid=2049544) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2049544) 'alpha': 0.5, (TaskRunner pid=2049544) 'distillation_add_tail': True, (TaskRunner pid=2049544) 'distillation_topk': 100, (TaskRunner pid=2049544) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2049544) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2049544) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'loss_weight': 0.0, (TaskRunner pid=2049544) 'mask': 'all'}, (TaskRunner pid=2049544) 'feedback_template': '\n' (TaskRunner pid=2049544) 'The ' (TaskRunner pid=2049544) 'following ' (TaskRunner pid=2049544) 'is ' (TaskRunner pid=2049544) 'feedback ' (TaskRunner pid=2049544) 'from ' (TaskRunner pid=2049544) 'your ' (TaskRunner pid=2049544) 'unsuccessful ' (TaskRunner pid=2049544) 'earlier ' (TaskRunner pid=2049544) 'attempt:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{feedback_raw}', (TaskRunner pid=2049544) 'full_logit_distillation': True, (TaskRunner pid=2049544) 'include_environment_feedback': True, (TaskRunner pid=2049544) 'is_clip': 2, (TaskRunner pid=2049544) 'max_reprompt_len': 22528, (TaskRunner pid=2049544) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2049544) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) 'Correctly ' (TaskRunner pid=2049544) 'solve ' (TaskRunner pid=2049544) 'the ' (TaskRunner pid=2049544) 'original ' (TaskRunner pid=2049544) 'question.', (TaskRunner pid=2049544) 'reprompt_truncation': 'right', (TaskRunner pid=2049544) 'solution_template': '\n' (TaskRunner pid=2049544) 'Correct ' (TaskRunner pid=2049544) 'solution:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{successful_previous_attempt}', (TaskRunner pid=2049544) 'srpo_dynamic_weighting': False, (TaskRunner pid=2049544) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2049544) 'success_reward_threshold': 0.5, (TaskRunner pid=2049544) 'teacher_regularization': 'ema', (TaskRunner pid=2049544) 'teacher_update_rate': 0.0, (TaskRunner pid=2049544) 'token_reweight_decay_steps': None, (TaskRunner pid=2049544) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2049544) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2049544) 'tau_neg': 1.05, (TaskRunner pid=2049544) 'tau_pos': 1.0, (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_kl_loss': False, (TaskRunner pid=2049544) 'use_prefix_grouper': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'hybrid_engine': True, (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2049544) 'custom_chat_template': None, (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'exclude_modules': None, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2049544) 'hf_config_path': None, (TaskRunner pid=2049544) 'lora_adapter_path': None, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2049544) 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': None, (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_liger': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'nccl_timeout': 600, (TaskRunner pid=2049544) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': True, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2049544) 'agent_loop_config_path': None, (TaskRunner pid=2049544) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2049544) 'name': None, (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2049544) 'num_workers': 8}, (TaskRunner pid=2049544) 'calculate_log_probs': True, (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enable_rollout_routing_replay': False, (TaskRunner pid=2049544) 'enforce_eager': False, (TaskRunner pid=2049544) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2049544) 'ignore_eos': False, (TaskRunner pid=2049544) 'layered_summon': False, (TaskRunner pid=2049544) 'load_format': 'dummy', (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'max_num_batched_tokens': 10240, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'mode': 'async', (TaskRunner pid=2049544) 'multi_stage_wake_up': False, (TaskRunner pid=2049544) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'format': 'hermes', (TaskRunner pid=2049544) 'interaction_config_path': None, (TaskRunner pid=2049544) 'max_assistant_turns': None, (TaskRunner pid=2049544) 'max_parallel_calls': 1, (TaskRunner pid=2049544) 'max_tool_response_length': 256, (TaskRunner pid=2049544) 'max_user_turns': None, (TaskRunner pid=2049544) 'num_repeat_rollouts': None, (TaskRunner pid=2049544) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2049544) 'use_inference_chat_template': False}, (TaskRunner pid=2049544) 'n': 8, (TaskRunner pid=2049544) 'name': 'vllm', (TaskRunner pid=2049544) 'over_sample_rate': 0, (TaskRunner pid=2049544) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2049544) 'port': 9090, (TaskRunner pid=2049544) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'quantization': None, (TaskRunner pid=2049544) 'quantization_config_file': None, (TaskRunner pid=2049544) 'response_length': 8192, (TaskRunner pid=2049544) 'scheduling_policy': 'fcfs', (TaskRunner pid=2049544) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2049544) 'skip_rollout': False, (TaskRunner pid=2049544) 'skip_tokenizer_init': True, (TaskRunner pid=2049544) 'temperature': 1.0, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 1.0, (TaskRunner pid=2049544) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2049544) 'backend': None, (TaskRunner pid=2049544) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2049544) 'token2text': False}, (TaskRunner pid=2049544) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2049544) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'n': 16, (TaskRunner pid=2049544) 'temperature': 0.6, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 0.95}}}, (TaskRunner pid=2049544) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2049544) 'adv_estimator': 'grpo', (TaskRunner pid=2049544) 'gamma': 1.0, (TaskRunner pid=2049544) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2049544) 'horizon': 10000, (TaskRunner pid=2049544) 'kl_coef': 0.001, (TaskRunner pid=2049544) 'target_kl': 0.1, (TaskRunner pid=2049544) 'type': 'fixed'}, (TaskRunner pid=2049544) 'kl_penalty': 'kl', (TaskRunner pid=2049544) 'lam': 1.0, (TaskRunner pid=2049544) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2049544) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2049544) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2049544) 'loss_type': 'ppo_clip', (TaskRunner pid=2049544) 'rollout_is': 'token', (TaskRunner pid=2049544) 'rollout_is_batch_normalize': False, (TaskRunner pid=2049544) 'rollout_is_threshold': 2.0, (TaskRunner pid=2049544) 'rollout_rs': None, (TaskRunner pid=2049544) 'rollout_rs_threshold': None}, (TaskRunner pid=2049544) 'use_kl_in_reward': False, (TaskRunner pid=2049544) 'use_pf_ppo': False}, (TaskRunner pid=2049544) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2049544) 'cliprange_value': 0.5, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'enable': None, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'forward_micro_batch_size': None, (TaskRunner pid=2049544) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-05, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': -1, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False}, (TaskRunner pid=2049544) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2049544) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2049544) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2049544) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'dataloader_num_workers': 8, (TaskRunner pid=2049544) 'filter_overlong_prompts': True, (TaskRunner pid=2049544) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2049544) 'image_key': 'images', (TaskRunner pid=2049544) 'image_patch_size': 14, (TaskRunner pid=2049544) 'max_prompt_length': 2048, (TaskRunner pid=2049544) 'max_response_length': 8192, (TaskRunner pid=2049544) 'prompt_key': 'prompt', (TaskRunner pid=2049544) 'return_full_prompt': False, (TaskRunner pid=2049544) 'return_multi_modal_inputs': True, (TaskRunner pid=2049544) 'return_raw_chat': True, (TaskRunner pid=2049544) 'return_raw_input_ids': False, (TaskRunner pid=2049544) 'reward_fn_key': 'data_source', (TaskRunner pid=2049544) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2049544) 'seed': None, (TaskRunner pid=2049544) 'shuffle': True, (TaskRunner pid=2049544) 'tokenizer': None, (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'train_batch_size': 32, (TaskRunner pid=2049544) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2049544) 'train_max_samples': 3200, (TaskRunner pid=2049544) 'truncation': 'error', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_shm': False, (TaskRunner pid=2049544) 'val_batch_size': None, (TaskRunner pid=2049544) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2049544) 'val_max_samples': -1, (TaskRunner pid=2049544) 'validation_shuffle': False, (TaskRunner pid=2049544) 'video_key': 'videos'}, (TaskRunner pid=2049544) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2049544) 'capture-range-end': None, (TaskRunner pid=2049544) 'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'kill': 'none', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2049544) 'torch_memory': {'context': 'all', (TaskRunner pid=2049544) 'kw_args': {}, (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'stacks': 'all', (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2049544) 'profile_continuous_steps': False, (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'steps': None, (TaskRunner pid=2049544) 'tool': None}, (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', (TaskRunner pid=2049544) 'include_dashboard': False, (TaskRunner pid=2049544) 'num_cpus': None, (TaskRunner pid=2049544) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'USER': 'root'}}}, (TaskRunner pid=2049544) 'timeline_json_file': None}, (TaskRunner pid=2049544) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2049544) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2049544) 'name': 'custom_reward_manager', (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'name': 'naive', (TaskRunner pid=2049544) 'source': 'register'}, (TaskRunner pid=2049544) 'reward_model': {'enable': False, (TaskRunner pid=2049544) 'enable_resource_pool': False, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'launch_reward_fn_async': False, (TaskRunner pid=2049544) 'max_length': None, (TaskRunner pid=2049544) 'micro_batch_size': None, (TaskRunner pid=2049544) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'model': {'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2049544) 'trust_remote_code': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 0, (TaskRunner pid=2049544) 'num_workers': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'reward_loop_class_name': None, (TaskRunner pid=2049544) 'reward_loop_module_path': None, (TaskRunner pid=2049544) 'reward_loop_source': 'register', (TaskRunner pid=2049544) 'reward_manager': 'naive', (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enforce_eager': True, (TaskRunner pid=2049544) 'engine_kwargs': {}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2049544) 'limit_images': None, (TaskRunner pid=2049544) 'load_format': 'auto', (TaskRunner pid=2049544) 'max_model_len': None, (TaskRunner pid=2049544) 'max_num_batched_tokens': 8192, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'name': '???', (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'response_length': 2048, (TaskRunner pid=2049544) 'skip_tokenizer_init': False, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2049544) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2049544) 'memory_limit_mb': 1024, (TaskRunner pid=2049544) 'url': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_reward_loop': False}, (TaskRunner pid=2049544) 'trainer': {'balance_batch': True, (TaskRunner pid=2049544) 'critic_warmup': 0, (TaskRunner pid=2049544) 'default_hdfs_dir': None, (TaskRunner pid=2049544) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'del_local_ckpt_after_load': False, (TaskRunner pid=2049544) 'device': 'cuda', (TaskRunner pid=2049544) 'esi_redundant_time': 0, (TaskRunner pid=2049544) 'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2049544) 'log_val_generations': 0, (TaskRunner pid=2049544) 'logger': ['console', 'wandb'], (TaskRunner pid=2049544) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2049544) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 1, (TaskRunner pid=2049544) 'project_name': 'SDPO-root', (TaskRunner pid=2049544) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2049544) 'resume_from_path': None, (TaskRunner pid=2049544) 'resume_mode': 'auto', (TaskRunner pid=2049544) 'rollout_data_dir': None, (TaskRunner pid=2049544) 'save_freq': 10, (TaskRunner pid=2049544) 'test_freq': 10, (TaskRunner pid=2049544) 'total_epochs': 30, (TaskRunner pid=2049544) 'total_training_steps': 100, (TaskRunner pid=2049544) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2049544) 'val_before_train': False, (TaskRunner pid=2049544) 'val_only': False, (TaskRunner pid=2049544) 'validation_data_dir': None}, (TaskRunner pid=2049544) 'transfer_queue': {'enable': False}, (TaskRunner pid=2049544) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'dir': '/users/root/SDPO', (TaskRunner pid=2049544) 'log_dir': '/users/root/output', (TaskRunner pid=2049544) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) use_critic=need_critic(config), (TaskRunner pid=2049544) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2049544) Using dataset class: RLHFDataset (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2049544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2049544) dataset len: 720 (TaskRunner pid=2049544) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) WARNING:2026-07-02 07:32:33,214:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2049544) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 121.99 examples/s] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) self.use_critic = need_critic(self.config) (WorkerDict pid=2049933) [W702 07:32:42.949021168 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049934) [W702 07:32:42.120202714 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2049932) Model config after override: Qwen3Config { (WorkerDict pid=2049932) "architectures": [ (WorkerDict pid=2049932) "Qwen3ForCausalLM" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "attention_bias": false, (WorkerDict pid=2049932) "attention_dropout": 0.0, (WorkerDict pid=2049932) "dtype": "bfloat16", (WorkerDict pid=2049932) "eos_token_id": 151645, (WorkerDict pid=2049932) "head_dim": 128, (WorkerDict pid=2049932) "hidden_act": "silu", (WorkerDict pid=2049932) "hidden_size": 2560, (WorkerDict pid=2049932) "initializer_range": 0.02, (WorkerDict pid=2049932) "intermediate_size": 9728, (WorkerDict pid=2049932) "layer_types": [ (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "max_position_embeddings": 40960, (WorkerDict pid=2049932) "max_window_layers": 36, (WorkerDict pid=2049932) "model_type": "qwen3", (WorkerDict pid=2049932) "num_attention_heads": 32, (WorkerDict pid=2049932) "num_hidden_layers": 36, (WorkerDict pid=2049932) "num_key_value_heads": 8, (WorkerDict pid=2049932) "pad_token_id": 151643, (WorkerDict pid=2049932) "rms_norm_eps": 1e-06, (WorkerDict pid=2049932) "rope_scaling": null, (WorkerDict pid=2049932) "rope_theta": 1000000, (WorkerDict pid=2049932) "sliding_window": null, (WorkerDict pid=2049932) "tie_word_embeddings": true, (WorkerDict pid=2049932) "transformers_version": "4.57.1", (WorkerDict pid=2049932) "use_cache": true, (WorkerDict pid=2049932) "use_sliding_window": false, (WorkerDict pid=2049932) "vocab_size": 151936 (WorkerDict pid=2049932) } (WorkerDict pid=2049932) (WorkerDict pid=2049932) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2049932) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049932) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2049932) Actor use_remove_padding=True (WorkerDict pid=2049932) Actor use_fused_kernels=False (WorkerDict pid=2049932) Actor use_prefix_grouper=False (WorkerDict pid=2049939) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2049939) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2049932) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049939) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.92s/it] [repeated 12x across cluster] (WorkerDict pid=2049939) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2049938) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.88s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.94s/it] [repeated 2x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2049933) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2049933) warnings.warn( (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2049932) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2050805) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050805) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050806) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050806) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050805) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2050805) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2050803) ['serve', (vLLMHttpServer pid=2050803) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2050803) '--dtype', (vLLMHttpServer pid=2050803) 'bfloat16', (vLLMHttpServer pid=2050803) '--load_format', (vLLMHttpServer pid=2050803) 'dummy', (vLLMHttpServer pid=2050803) '--trust_remote_code', (vLLMHttpServer pid=2050803) '--max_model_len', (vLLMHttpServer pid=2050803) '40960', (vLLMHttpServer pid=2050803) '--max_num_seqs', (vLLMHttpServer pid=2050803) '1024', (vLLMHttpServer pid=2050803) '--enable_chunked_prefill', (vLLMHttpServer pid=2050803) '--max_num_batched_tokens', (vLLMHttpServer pid=2050803) '10240', (vLLMHttpServer pid=2050803) '--enable_prefix_caching', (vLLMHttpServer pid=2050803) '--enable_sleep_mode', (vLLMHttpServer pid=2050803) '--logprobs_mode', (vLLMHttpServer pid=2050803) 'processed_logprobs', (vLLMHttpServer pid=2050803) '--disable_custom_all_reduce', (vLLMHttpServer pid=2050803) '--gpu_memory_utilization', (vLLMHttpServer pid=2050803) '0.8', (vLLMHttpServer pid=2050803) '--disable_log_stats', (vLLMHttpServer pid=2050803) '--tensor_parallel_size', (vLLMHttpServer pid=2050803) '2', (vLLMHttpServer pid=2050803) '--seed', (vLLMHttpServer pid=2050803) '0', (vLLMHttpServer pid=2050803) '--override_generation_config', (vLLMHttpServer pid=2050803) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2050803) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2050803) '--hf_overrides', (vLLMHttpServer pid=2050803) '{}', (vLLMHttpServer pid=2050803) '--scheduling_policy', (vLLMHttpServer pid=2050803) 'fcfs'] (WorkerDict pid=2049938) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2050805) WARNING 07-02 07:33:26 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2049936) WARNING 07-02 07:33:34 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2050803) WARNING 07-02 07:33:27 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2049934) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049934) 2026-07-02 07:33:49,587 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2049934) 2026-07-02 07:33:49,606 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2050803) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2050803) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2050803) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2050803) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substituting the given values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is B: 56 pC. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2049544) step:10 - global_seqlen/min:15442 - global_seqlen/max:25018 - global_seqlen/minmax_diff:9576 - global_seqlen/balanced_min:21213 - global_seqlen/balanced_max:21224 - global_seqlen/mean:21220.5 - actor/entropy:0.12735222280025482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40465062856674194 - training/rollout_probs_diff_mean:0.0038589579053223133 - training/rollout_probs_diff_std:0.013698565773665905 - training/rollout_actor_probs_pearson_corr:0.9970982670783997 - rollout_corr/rollout_is_mean:1.0000947713851929 - rollout_corr/rollout_is_ratio_fraction_high:3.965264113503508e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020817638142034411 - rollout_corr/rollout_is_max:3.167215347290039 - rollout_corr/rollout_is_min:0.19278468191623688 - rollout_corr/rollout_is_std:0.03876073658466339 - rollout_corr/rollout_is_eff_sample_size:0.9985000968313013 - rollout_corr/rollout_is_seq_mean:1.000048041343689 - rollout_corr/rollout_is_seq_std:0.002764123724773526 - rollout_corr/rollout_is_seq_max:1.0100802183151245 - rollout_corr/rollout_is_seq_min:0.9903356432914734 - rollout_corr/rollout_is_seq_max_deviation:0.010080218315124512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1353368978016078) - rollout_corr/training_log_ppl:np.float64(0.12389787597203394) - rollout_corr/kl:np.float64(0.000877203190441378) - rollout_corr/k3_kl:np.float64(0.0008888360377170557) - rollout_corr/rollout_ppl:np.float64(1.134331472683698) - rollout_corr/rollout_log_ppl:np.float64(0.12302067201380851) - rollout_corr/log_ppl_diff:np.float64(0.0008772039582254365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020878618815913796) - rollout_corr/log_ppl_diff_max:np.float64(0.012568935751914978) - rollout_corr/log_ppl_diff_min:np.float64(-0.007140792906284332) - rollout_corr/ppl_ratio:np.float64(1.000881330575794) - rollout_corr/chi2_token:np.float64(0.0018501763697713614) - rollout_corr/chi2_seq:np.float64(0.666111497906968) - actor/pg_loss:np.float64(9.953684639185667e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005019576683480409) - actor/ppo_kl:np.float64(2.3842763525294686e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3116539753973484) - perf/mfu/actor:np.float64(0.0619170865443955) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(98.92141723632812) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/score/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/score/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07429798598036827) - val-core/sciknoweval/acc/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1013687608239918) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.777675) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.047588411442781434) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6271749999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.015128841196122723) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0035548642615965665) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.992525) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0202257687927191) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.014896175746267976) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.986025) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.026012467615322592) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.998875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.008222303730994859) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9748249999999998) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.030646040504587458) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0039815903242650785) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9594374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.02978871055390577) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0010787124614257437) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008162248879671097 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008162248879671097 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.046875 - response_length/max:1270.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.046875 - response_length_non_aborted/max:1270.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013142451643943787 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941465875133872) - timing_s/agent_loop/generate_sequences/max:np.float64(8.937132587656379) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.961100311316841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.937132587656379) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1270 - timing_s/gen:14.806582853198051 - timing_s/reward:0.059918224811553955 - timing_s/old_log_prob:2.401917176321149 - timing_s/adv:0.15370049141347408 - timing_s/update_actor:9.923850152641535 - timing_s/step:27.43245600350201 - timing_s/testing:88.77899980731308 - timing_s/save_checkpoint:6.549710331484675 - timing_s/stop_profile:0.00011613219976425171 - timing_per_token_ms/update_actor:0.058456740844004235 - timing_per_token_ms/gen:0.14678003542168655 - timing_per_token_ms/adv:0.0009053774146077736 - perf/total_num_tokens:169764 - perf/time_per_step:27.43245600350201 - perf/throughput:773.5545077440753 (TaskRunner pid=2049544) Training Progress: 10%|█ | 10/100 [06:47<1:27:44, 58.49s/it] (TaskRunner pid=2049544) step:11 - global_seqlen/min:16821 - global_seqlen/max:31114 - global_seqlen/minmax_diff:14293 - global_seqlen/balanced_min:23187 - global_seqlen/balanced_max:29684 - global_seqlen/mean:24008.625 - actor/entropy:0.11019539088010788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948736071586609 - training/rollout_probs_diff_mean:0.003139072796329856 - training/rollout_probs_diff_std:0.012617148458957672 - training/rollout_actor_probs_pearson_corr:0.997302770614624 - rollout_corr/rollout_is_mean:1.0000996589660645 - rollout_corr/rollout_is_ratio_fraction_high:7.389344682451338e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.852459334069863e-05 - rollout_corr/rollout_is_max:4.887231826782227 - rollout_corr/rollout_is_min:0.26104938983917236 - rollout_corr/rollout_is_std:0.03467484563589096 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:1.0002052783966064 - rollout_corr/rollout_is_seq_std:0.0023445014376193285 - rollout_corr/rollout_is_seq_max:1.0106254816055298 - rollout_corr/rollout_is_seq_min:0.99307781457901 - rollout_corr/rollout_is_seq_max_deviation:0.010625481605529785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1195416245609522) - rollout_corr/training_log_ppl:np.float64(0.1096741780929733) - rollout_corr/kl:np.float64(0.0007273018066071835) - rollout_corr/k3_kl:np.float64(0.0009105393462505162) - rollout_corr/rollout_ppl:np.float64(1.1187340491451323) - rollout_corr/rollout_log_ppl:np.float64(0.10894687551262905) - rollout_corr/log_ppl_diff:np.float64(0.0007273025803442579) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019026359186682384) - rollout_corr/log_ppl_diff_max:np.float64(0.022861547768115997) - rollout_corr/log_ppl_diff_min:np.float64(-0.007030755281448364) - rollout_corr/ppl_ratio:np.float64(1.000731429317966) - rollout_corr/chi2_token:np.float64(0.002295879879966378) - rollout_corr/chi2_seq:np.float64(2.1700741790700704) - actor/pg_loss:np.float64(8.732720743864775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011307528586712579) - actor/ppo_kl:np.float64(0.00020700877354862257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.46738822013139725) - perf/mfu/actor:np.float64(0.06503294604815439) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.65916442871094) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01939600333571434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01939600333571434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.76953125 - response_length/max:8192.0 - response_length/min:69.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:475.76953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001276358962059021 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8453756030648947) - timing_s/agent_loop/generate_sequences/max:np.float64(51.0958620775491) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.681928445308586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.0958620775491) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.71189962141216 - timing_s/reward:0.06205694004893303 - timing_s/old_log_prob:2.802142111584544 - timing_s/adv:0.1481334324926138 - timing_s/update_actor:10.832650827243924 - timing_s/step:70.63796724379063 - timing_s/stop_profile:0.00011838413774967194 - timing_per_token_ms/update_actor:0.05639978771818421 - timing_per_token_ms/gen:0.46562640805120126 - timing_per_token_ms/adv:0.0007712511258590079 - perf/total_num_tokens:192069 - perf/time_per_step:70.63796724379063 - perf/throughput:339.8827278981539 (TaskRunner pid=2049544) Training Progress: 11%|█ | 11/100 [07:58<1:32:18, 62.23s/it] (TaskRunner pid=2049544) step:12 - global_seqlen/min:18163 - global_seqlen/max:27095 - global_seqlen/minmax_diff:8932 - global_seqlen/balanced_min:23218 - global_seqlen/balanced_max:23238 - global_seqlen/mean:23234.125 - actor/entropy:0.11550334095954895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4828122854232788 - training/rollout_probs_diff_mean:0.0033159777522087097 - training/rollout_probs_diff_std:0.012482362799346447 - training/rollout_actor_probs_pearson_corr:0.9973734021186829 - rollout_corr/rollout_is_mean:0.9999693036079407 - rollout_corr/rollout_is_ratio_fraction_high:5.898561721551232e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010111819574376568 - rollout_corr/rollout_is_max:2.5878732204437256 - rollout_corr/rollout_is_min:0.125954732298851 - rollout_corr/rollout_is_std:0.03412209078669548 - rollout_corr/rollout_is_eff_sample_size:0.9988369179946218 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0017772279679775238 - rollout_corr/rollout_is_seq_max:1.008151888847351 - rollout_corr/rollout_is_seq_min:0.9952659010887146 - rollout_corr/rollout_is_seq_max_deviation:0.008151888847351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1121750511229038) - rollout_corr/training_log_ppl:np.float64(0.10409450641964213) - rollout_corr/kl:np.float64(0.0006273885898906428) - rollout_corr/k3_kl:np.float64(0.000663624501786586) - rollout_corr/rollout_ppl:np.float64(1.1114575085230172) - rollout_corr/rollout_log_ppl:np.float64(0.10346711749298265) - rollout_corr/log_ppl_diff:np.float64(0.0006273889266594779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014779563243791927) - rollout_corr/log_ppl_diff_max:np.float64(0.006399750709533691) - rollout_corr/log_ppl_diff_min:np.float64(-0.006532169878482819) - rollout_corr/ppl_ratio:np.float64(1.000629240879789) - rollout_corr/chi2_token:np.float64(0.0014743746723979712) - rollout_corr/chi2_seq:np.float64(1.0935420689638704) - actor/pg_loss:np.float64(0.00015071697998791933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004757110639275197) - actor/ppo_kl:np.float64(2.2654555305479107e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2736458256840706) - perf/mfu/actor:np.float64(0.0671891188671343) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.26230239868164) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065000043250620365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065000043250620365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.56640625 - response_length/max:1365.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.56640625 - response_length_non_aborted/max:1365.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.028241038322449e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0592438988387585) - timing_s/agent_loop/generate_sequences/max:np.float64(10.160819062963128) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8243368406328955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.160819062963128) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1365 - timing_s/gen:15.931704020127654 - timing_s/reward:0.06278514675796032 - timing_s/old_log_prob:2.4699926115572453 - timing_s/adv:0.16309965774416924 - timing_s/update_actor:10.090087844058871 - timing_s/step:28.80461708456278 - timing_s/stop_profile:0.0001237243413925171 - timing_per_token_ms/update_actor:0.054284849569646326 - timing_per_token_ms/gen:0.13424876779155878 - timing_per_token_ms/adv:0.0008774790192452332 - perf/total_num_tokens:185873 - perf/time_per_step:28.80461708456278 - perf/throughput:806.6111391722626 (TaskRunner pid=2049544) Training Progress: 12%|█▏ | 12/100 [08:26<1:16:22, 52.08s/it] (TaskRunner pid=2049544) step:13 - global_seqlen/min:20734 - global_seqlen/max:28493 - global_seqlen/minmax_diff:7759 - global_seqlen/balanced_min:23965 - global_seqlen/balanced_max:23977 - global_seqlen/mean:23970.125 - actor/entropy:0.1322651505470276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5841533541679382 - training/rollout_probs_diff_mean:0.003578777192160487 - training/rollout_probs_diff_std:0.01281787734478712 - training/rollout_actor_probs_pearson_corr:0.997634768486023 - rollout_corr/rollout_is_mean:0.9998666048049927 - rollout_corr/rollout_is_ratio_fraction_high:8.307234566018451e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010799404844874516 - rollout_corr/rollout_is_max:2.1130197048187256 - rollout_corr/rollout_is_min:0.289154976606369 - rollout_corr/rollout_is_std:0.03564285859465599 - rollout_corr/rollout_is_eff_sample_size:0.9987307228760026 - rollout_corr/rollout_is_seq_mean:0.9998607635498047 - rollout_corr/rollout_is_seq_std:0.0017514645587652922 - rollout_corr/rollout_is_seq_max:1.0048136711120605 - rollout_corr/rollout_is_seq_min:0.9948503971099854 - rollout_corr/rollout_is_seq_max_deviation:0.0051496028900146484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1441420195624232) - rollout_corr/training_log_ppl:np.float64(0.12973640582640655) - rollout_corr/kl:np.float64(0.0009200233825930049) - rollout_corr/k3_kl:np.float64(0.000771395374357553) - rollout_corr/rollout_ppl:np.float64(1.1430599559098482) - rollout_corr/rollout_log_ppl:np.float64(0.12881638239923632) - rollout_corr/log_ppl_diff:np.float64(0.0009200234271702357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016153893011505716) - rollout_corr/log_ppl_diff_max:np.float64(0.010122627019882202) - rollout_corr/log_ppl_diff_min:np.float64(-0.007734514772891998) - rollout_corr/ppl_ratio:np.float64(1.0009225921239704) - rollout_corr/chi2_token:np.float64(0.0012212155852466822) - rollout_corr/chi2_seq:np.float64(0.878853014903143) - actor/pg_loss:np.float64(-4.01295255869627e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007891653215210681) - actor/ppo_kl:np.float64(0.00011588802720652325) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40201497077941895) - perf/mfu/actor:np.float64(0.0691041251464859) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.31333541870117) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005807795561850071 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005807795561850071 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.22265625 - response_length/max:1333.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.22265625 - response_length_non_aborted/max:1333.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014058127999305725 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4221638720482588) - timing_s/agent_loop/generate_sequences/max:np.float64(9.880332147702575) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.85110625057132) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.880332147702575) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1333 - timing_s/gen:15.408700393512845 - timing_s/reward:0.062333619222044945 - timing_s/old_log_prob:2.5665055103600025 - timing_s/adv:0.1314475554972887 - timing_s/update_actor:10.066378388553858 - timing_s/step:28.321861196309328 - timing_s/stop_profile:0.00011406466364860535 - timing_per_token_ms/update_actor:0.05249439869709616 - timing_per_token_ms/gen:0.12800369168124182 - timing_per_token_ms/adv:0.0006854759596439772 - perf/total_num_tokens:191761 - perf/time_per_step:28.321861196309328 - perf/throughput:846.3470968187496 (TaskRunner pid=2049544) Training Progress: 13%|█▎ | 13/100 [08:55<1:05:06, 44.90s/it] (TaskRunner pid=2049544) step:14 - global_seqlen/min:17304 - global_seqlen/max:31074 - global_seqlen/minmax_diff:13770 - global_seqlen/balanced_min:23194 - global_seqlen/balanced_max:23602 - global_seqlen/mean:23258.875 - actor/entropy:0.11846989393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45436424016952515 - training/rollout_probs_diff_mean:0.003607378341257572 - training/rollout_probs_diff_std:0.013258584775030613 - training/rollout_actor_probs_pearson_corr:0.9971495866775513 - rollout_corr/rollout_is_mean:0.9998986721038818 - rollout_corr/rollout_is_ratio_fraction_high:5.1646224164869636e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014633095997851342 - rollout_corr/rollout_is_max:2.4299769401550293 - rollout_corr/rollout_is_min:0.09816896170377731 - rollout_corr/rollout_is_std:0.037202417850494385 - rollout_corr/rollout_is_eff_sample_size:0.9986175363719597 - rollout_corr/rollout_is_seq_mean:0.9998643398284912 - rollout_corr/rollout_is_seq_std:0.002263482892885804 - rollout_corr/rollout_is_seq_max:1.00752854347229 - rollout_corr/rollout_is_seq_min:0.9923214316368103 - rollout_corr/rollout_is_seq_max_deviation:0.007678568363189697 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.13216158375144) - rollout_corr/training_log_ppl:np.float64(0.12105548306863056) - rollout_corr/kl:np.float64(0.0011141863628356674) - rollout_corr/k3_kl:np.float64(0.0007862591612095571) - rollout_corr/rollout_ppl:np.float64(1.1308507332578301) - rollout_corr/rollout_log_ppl:np.float64(0.1199412965834199) - rollout_corr/log_ppl_diff:np.float64(0.0011141864852106664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018289975014340598) - rollout_corr/log_ppl_diff_max:np.float64(0.01189817488193512) - rollout_corr/log_ppl_diff_min:np.float64(-0.006560362875461578) - rollout_corr/ppl_ratio:np.float64(1.0011174657847732) - rollout_corr/chi2_token:np.float64(0.0008673113770782948) - rollout_corr/chi2_seq:np.float64(0.27274479530751705) - actor/pg_loss:np.float64(8.668552618473768e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010782241606648313) - actor/ppo_kl:np.float64(0.00016768844109904535) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42187249660491943) - perf/mfu/actor:np.float64(0.06630021670401551) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.41584777832031) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009940821677446365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009940821677446365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.80859375 - response_length/max:1963.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.80859375 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013650394976139069 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0465375203639269) - timing_s/agent_loop/generate_sequences/max:np.float64(12.73308464512229) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.650266676486353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.73308464512229) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.954351030290127 - timing_s/reward:0.06252953223884106 - timing_s/old_log_prob:2.473697155714035 - timing_s/adv:0.14756855927407742 - timing_s/update_actor:9.925982998684049 - timing_s/step:31.649879788979888 - timing_s/stop_profile:0.00011733733117580414 - timing_per_token_ms/update_actor:0.05334513706426068 - timing_per_token_ms/gen:0.16315344119036046 - timing_per_token_ms/adv:0.0007930766173884023 - perf/total_num_tokens:186071 - perf/time_per_step:31.649879788979888 - perf/throughput:734.8803583165097 (TaskRunner pid=2049544) Training Progress: 14%|█▍ | 14/100 [09:26<58:38, 40.91s/it] (TaskRunner pid=2049544) step:15 - global_seqlen/min:15342 - global_seqlen/max:29838 - global_seqlen/minmax_diff:14496 - global_seqlen/balanced_min:23634 - global_seqlen/balanced_max:23672 - global_seqlen/mean:23657.75 - actor/entropy:0.1110260859131813 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9913476705551147 - training/rollout_probs_diff_mean:0.0030704238452017307 - training/rollout_probs_diff_std:0.01288246363401413 - training/rollout_actor_probs_pearson_corr:0.9973083138465881 - rollout_corr/rollout_is_mean:0.9999945163726807 - rollout_corr/rollout_is_ratio_fraction_high:3.303655466879718e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.085052442969754e-05 - rollout_corr/rollout_is_max:3.397390604019165 - rollout_corr/rollout_is_min:0.008578135631978512 - rollout_corr/rollout_is_std:0.03370378911495209 - rollout_corr/rollout_is_eff_sample_size:0.9988653435875123 - rollout_corr/rollout_is_seq_mean:0.9999395608901978 - rollout_corr/rollout_is_seq_std:0.0021285165566951036 - rollout_corr/rollout_is_seq_max:1.016195297241211 - rollout_corr/rollout_is_seq_min:0.9913737177848816 - rollout_corr/rollout_is_seq_max_deviation:0.016195297241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116935707628727) - rollout_corr/training_log_ppl:np.float64(0.10718949149304535) - rollout_corr/kl:np.float64(0.0008361445555777891) - rollout_corr/k3_kl:np.float64(0.0006998892279455049) - rollout_corr/rollout_ppl:np.float64(1.1159698073752224) - rollout_corr/rollout_log_ppl:np.float64(0.10635334601101931) - rollout_corr/log_ppl_diff:np.float64(0.0008361454820260406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016929644480114803) - rollout_corr/log_ppl_diff_max:np.float64(0.01026906818151474) - rollout_corr/log_ppl_diff_min:np.float64(-0.012994319200515747) - rollout_corr/ppl_ratio:np.float64(1.000839062500745) - rollout_corr/chi2_token:np.float64(0.0011396266054362059) - rollout_corr/chi2_seq:np.float64(1.5898604474496096) - actor/pg_loss:np.float64(-2.7647125534713268e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006954610362299718) - actor/ppo_kl:np.float64(0.00011087026399003719) - actor/pg_clipfrac_lower:np.float64(1.0444518920849077e-05) - actor/grad_norm:np.float64(0.29682732000947) - perf/mfu/actor:np.float64(0.06723820699139785) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.3052020072937) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0049513536505401134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0049513536505401134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.9609375 - response_length/max:1790.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.9609375 - response_length_non_aborted/max:1790.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010203756392002106 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7883450742810965) - timing_s/agent_loop/generate_sequences/max:np.float64(12.01114228926599) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.683357301946671) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.01114228926599) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1790 - timing_s/gen:17.793763728812337 - timing_s/reward:0.06301617436110973 - timing_s/old_log_prob:2.4648690931499004 - timing_s/adv:0.15438295900821686 - timing_s/update_actor:10.23603860847652 - timing_s/step:30.79816811159253 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/update_actor:0.05408396090327969 - timing_per_token_ms/gen:0.14696116328988204 - timing_per_token_ms/adv:0.0008157102799728253 - perf/total_num_tokens:189262 - perf/time_per_step:30.79816811159253 - perf/throughput:768.1544536765856 (TaskRunner pid=2049544) Training Progress: 15%|█▌ | 15/100 [09:57<53:39, 37.88s/it] (TaskRunner pid=2049544) step:16 - global_seqlen/min:18400 - global_seqlen/max:27667 - global_seqlen/minmax_diff:9267 - global_seqlen/balanced_min:22148 - global_seqlen/balanced_max:22154 - global_seqlen/mean:22151.125 - actor/entropy:0.12429671734571457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.739967942237854 - training/rollout_probs_diff_mean:0.0033937261905521154 - training/rollout_probs_diff_std:0.012556870467960835 - training/rollout_actor_probs_pearson_corr:0.9975789189338684 - rollout_corr/rollout_is_mean:1.000154972076416 - rollout_corr/rollout_is_ratio_fraction_high:2.7087302441941574e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.126190368784592e-05 - rollout_corr/rollout_is_max:4.322726726531982 - rollout_corr/rollout_is_min:0.12595464289188385 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987801905807853 - rollout_corr/rollout_is_seq_mean:1.0001484155654907 - rollout_corr/rollout_is_seq_std:0.0019322986481711268 - rollout_corr/rollout_is_seq_max:1.007652997970581 - rollout_corr/rollout_is_seq_min:0.992692768573761 - rollout_corr/rollout_is_seq_max_deviation:0.007652997970581055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.128569976426661) - rollout_corr/training_log_ppl:np.float64(0.11689590534660965) - rollout_corr/kl:np.float64(0.0006918034323035727) - rollout_corr/k3_kl:np.float64(0.0007205118381676812) - rollout_corr/rollout_ppl:np.float64(1.1277636121958494) - rollout_corr/rollout_log_ppl:np.float64(0.11620410031173378) - rollout_corr/log_ppl_diff:np.float64(0.0006918050348758698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016668791504343972) - rollout_corr/log_ppl_diff_max:np.float64(0.011440090835094452) - rollout_corr/log_ppl_diff_min:np.float64(-0.005701005458831787) - rollout_corr/ppl_ratio:np.float64(1.000694399466738) - rollout_corr/chi2_token:np.float64(0.0014990693889558315) - rollout_corr/chi2_seq:np.float64(0.6086118693929166) - actor/pg_loss:np.float64(3.552588168531656e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008225217134167906) - actor/ppo_kl:np.float64(0.0001884127358140475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3684859052300453) - perf/mfu/actor:np.float64(0.05525549139638902) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(118.55257987976074) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014105712994933128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014105712994933128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:432.62890625 - response_length/max:1203.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.62890625 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.934604167938232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0808530896902084) - timing_s/agent_loop/generate_sequences/max:np.float64(8.854041801765561) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.365803626016714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.854041801765561) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1203 - timing_s/gen:14.700905803591013 - timing_s/reward:0.06069855950772762 - timing_s/old_log_prob:2.4508451614528894 - timing_s/adv:0.14457714557647705 - timing_s/update_actor:11.594925744459033 - timing_s/step:29.03880488872528 - timing_s/stop_profile:0.00011275149881839752 - timing_per_token_ms/update_actor:0.0654307949622143 - timing_per_token_ms/gen:0.13273596023214732 - timing_per_token_ms/adv:0.0008158566753182798 - perf/total_num_tokens:177209 - perf/time_per_step:29.03880488872528 - perf/throughput:762.8111792093924 (TaskRunner pid=2049544) Training Progress: 16%|█▌ | 16/100 [10:26<49:20, 35.25s/it] (TaskRunner pid=2049544) step:17 - global_seqlen/min:17199 - global_seqlen/max:28844 - global_seqlen/minmax_diff:11645 - global_seqlen/balanced_min:22404 - global_seqlen/balanced_max:28744 - global_seqlen/mean:23204.125 - actor/entropy:0.12878382205963135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7316876649856567 - training/rollout_probs_diff_mean:0.003660086076706648 - training/rollout_probs_diff_std:0.013964627869427204 - training/rollout_actor_probs_pearson_corr:0.9971086978912354 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:5.108600453240797e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000221372683881782 - rollout_corr/rollout_is_max:3.039975881576538 - rollout_corr/rollout_is_min:0.002080337144434452 - rollout_corr/rollout_is_std:0.037413306534290314 - rollout_corr/rollout_is_eff_sample_size:0.9986019633462979 - rollout_corr/rollout_is_seq_mean:0.9999227523803711 - rollout_corr/rollout_is_seq_std:0.0023962773848325014 - rollout_corr/rollout_is_seq_max:1.0085341930389404 - rollout_corr/rollout_is_seq_min:0.9915060997009277 - rollout_corr/rollout_is_seq_max_deviation:0.00853419303894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1416860399767756) - rollout_corr/training_log_ppl:np.float64(0.1297323260405392) - rollout_corr/kl:np.float64(0.0012029753426432421) - rollout_corr/k3_kl:np.float64(0.0010000968625991646) - rollout_corr/rollout_ppl:np.float64(1.1402671793475747) - rollout_corr/rollout_log_ppl:np.float64(0.12852934912734781) - rollout_corr/log_ppl_diff:np.float64(0.0012029769131913781) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022540156060131267) - rollout_corr/log_ppl_diff_max:np.float64(0.022689037024974823) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066199153661727905) - rollout_corr/ppl_ratio:np.float64(1.0012089894153178) - rollout_corr/chi2_token:np.float64(0.0014632882084697485) - rollout_corr/chi2_seq:np.float64(0.5507866519037634) - actor/pg_loss:np.float64(0.0002590372459962964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013125582909196964) - actor/ppo_kl:np.float64(0.00025115833264877097) - actor/pg_clipfrac_lower:np.float64(7.70463520893827e-06) - actor/grad_norm:np.float64(0.3952488452196121) - perf/mfu/actor:np.float64(0.06094650606696392) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(110.15214920043945) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022923780605196953 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022923780605196953 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.78515625 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:458.78515625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017055682837963104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8617520779371262) - timing_s/agent_loop/generate_sequences/max:np.float64(50.32327074185014) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.232911160150252) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.32327074185014) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.11839736439288 - timing_s/reward:0.06247155740857124 - timing_s/old_log_prob:2.831888886168599 - timing_s/adv:0.1662355735898018 - timing_s/update_actor:10.9862146563828 - timing_s/step:71.25081997551024 - timing_s/stop_profile:0.00011354684829711914 - timing_per_token_ms/update_actor:0.05918244415800423 - timing_per_token_ms/gen:0.4863251059131442 - timing_per_token_ms/adv:0.0008955065833650363 - perf/total_num_tokens:185633 - perf/time_per_step:71.25081997551024 - perf/throughput:325.6681818956685 (TaskRunner pid=2049544) Training Progress: 17%|█▋ | 17/100 [11:38<1:03:45, 46.09s/it] (TaskRunner pid=2049544) step:18 - global_seqlen/min:17313 - global_seqlen/max:30331 - global_seqlen/minmax_diff:13018 - global_seqlen/balanced_min:25566 - global_seqlen/balanced_max:25574 - global_seqlen/mean:25570.75 - actor/entropy:0.1224943995475769 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5235333442687988 - training/rollout_probs_diff_mean:0.0032062120735645294 - training/rollout_probs_diff_std:0.012459194287657738 - training/rollout_actor_probs_pearson_corr:0.9975793361663818 - rollout_corr/rollout_is_mean:0.9999059438705444 - rollout_corr/rollout_is_ratio_fraction_high:7.298523996723816e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012407491158228368 - rollout_corr/rollout_is_max:2.7950527667999268 - rollout_corr/rollout_is_min:0.06372947990894318 - rollout_corr/rollout_is_std:0.03364891931414604 - rollout_corr/rollout_is_eff_sample_size:0.9988689117678543 - rollout_corr/rollout_is_seq_mean:0.9998814463615417 - rollout_corr/rollout_is_seq_std:0.0018376336665824056 - rollout_corr/rollout_is_seq_max:1.007022738456726 - rollout_corr/rollout_is_seq_min:0.9937419891357422 - rollout_corr/rollout_is_seq_max_deviation:0.007022738456726074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1279504764825106) - rollout_corr/training_log_ppl:np.float64(0.11757075608329615) - rollout_corr/kl:np.float64(0.0008173328948331005) - rollout_corr/k3_kl:np.float64(0.0006817538003787149) - rollout_corr/rollout_ppl:np.float64(1.1270086439326406) - rollout_corr/rollout_log_ppl:np.float64(0.1167534222076938) - rollout_corr/log_ppl_diff:np.float64(0.000817333875602344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016419640196545515) - rollout_corr/log_ppl_diff_max:np.float64(0.008472427725791931) - rollout_corr/log_ppl_diff_min:np.float64(-0.006251677870750427) - rollout_corr/ppl_ratio:np.float64(1.0008197950664908) - rollout_corr/chi2_token:np.float64(0.0010528604034334421) - rollout_corr/chi2_seq:np.float64(0.8530804882757366) - actor/pg_loss:np.float64(0.0001363652991130948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009771198135695158) - actor/ppo_kl:np.float64(8.36708865672442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4684727042913437) - perf/mfu/actor:np.float64(0.0710728954199909) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.35284042358398) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004924679175019264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004924679175019264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.2109375 - response_length/max:1581.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.2109375 - response_length_non_aborted/max:1581.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012665800750255585 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8409140948206186) - timing_s/agent_loop/generate_sequences/max:np.float64(11.426093036308885) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7903019126315485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.426093036308885) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1581 - timing_s/gen:17.257411571219563 - timing_s/reward:0.06468412093818188 - timing_s/old_log_prob:2.512243326753378 - timing_s/adv:0.16402030177414417 - timing_s/update_actor:10.535967061296105 - timing_s/step:30.621809538453817 - timing_s/stop_profile:0.00020102225244045258 - timing_per_token_ms/update_actor:0.05150399900910271 - timing_per_token_ms/gen:0.1259536366445733 - timing_per_token_ms/adv:0.000801796494892329 - perf/total_num_tokens:204566 - perf/time_per_step:30.621809538453817 - perf/throughput:835.0502594527971 (TaskRunner pid=2049544) Training Progress: 18%|█▊ | 18/100 [12:08<56:39, 41.46s/it] (TaskRunner pid=2049544) step:19 - global_seqlen/min:17713 - global_seqlen/max:31095 - global_seqlen/minmax_diff:13382 - global_seqlen/balanced_min:23780 - global_seqlen/balanced_max:23986 - global_seqlen/mean:23818.0 - actor/entropy:0.10652168840169907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43985795974731445 - training/rollout_probs_diff_mean:0.003031388856470585 - training/rollout_probs_diff_std:0.012130273506045341 - training/rollout_actor_probs_pearson_corr:0.9975038766860962 - rollout_corr/rollout_is_mean:1.000024437904358 - rollout_corr/rollout_is_ratio_fraction_high:5.6943903473438695e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010575295891612768 - rollout_corr/rollout_is_max:3.544985771179199 - rollout_corr/rollout_is_min:0.2985078692436218 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988546391994405 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.002387973479926586 - rollout_corr/rollout_is_seq_max:1.0142525434494019 - rollout_corr/rollout_is_seq_min:0.9915294051170349 - rollout_corr/rollout_is_seq_max_deviation:0.014252543449401855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1093668043613434) - rollout_corr/training_log_ppl:np.float64(0.10132705738578807) - rollout_corr/kl:np.float64(0.0006060766052620181) - rollout_corr/k3_kl:np.float64(0.0007317086579945453) - rollout_corr/rollout_ppl:np.float64(1.1086922818794847) - rollout_corr/rollout_log_ppl:np.float64(0.10072098037198884) - rollout_corr/log_ppl_diff:np.float64(0.0006060770137992222) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016289143422909547) - rollout_corr/log_ppl_diff_max:np.float64(0.014358222484588623) - rollout_corr/log_ppl_diff_min:np.float64(-0.009524352848529816) - rollout_corr/ppl_ratio:np.float64(1.0006090987008065) - rollout_corr/chi2_token:np.float64(0.001838616793975234) - rollout_corr/chi2_seq:np.float64(0.987432076362893) - actor/pg_loss:np.float64(0.0002443152479827404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007472785102891066) - actor/ppo_kl:np.float64(-8.193314219084868e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4798855558037758) - perf/mfu/actor:np.float64(0.0686439488762442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.47739028930664) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009253384196199477 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009253384196199477 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.1875 - response_length/max:1871.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.1875 - response_length_non_aborted/max:1871.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001460295170545578 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0999032072722912) - timing_s/agent_loop/generate_sequences/max:np.float64(12.776574049144983) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.886515708894876) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.776574049144983) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1871 - timing_s/gen:18.46205241046846 - timing_s/reward:0.06260044127702713 - timing_s/old_log_prob:2.540384005755186 - timing_s/adv:0.14328701607882977 - timing_s/update_actor:10.04507009498775 - timing_s/step:31.340227657929063 - timing_s/stop_profile:0.00011619366705417633 - timing_per_token_ms/update_actor:0.05271785044392765 - timing_per_token_ms/gen:0.15018590077499397 - timing_per_token_ms/adv:0.0007519891262848989 - perf/total_num_tokens:190544 - perf/time_per_step:31.340227657929063 - perf/throughput:759.9817161498524 (TaskRunner pid=2049544) Training Progress: 19%|█▉ | 19/100 [12:40<51:52, 38.43s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are asked to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2049544) step:20 - global_seqlen/min:17554 - global_seqlen/max:28214 - global_seqlen/minmax_diff:10660 - global_seqlen/balanced_min:22414 - global_seqlen/balanced_max:22431 - global_seqlen/mean:22418.0 - actor/entropy:0.11887378990650177 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7652756571769714 - training/rollout_probs_diff_mean:0.003253082511946559 - training/rollout_probs_diff_std:0.01256453339010477 - training/rollout_actor_probs_pearson_corr:0.9974572062492371 - rollout_corr/rollout_is_mean:0.9998491406440735 - rollout_corr/rollout_is_ratio_fraction_high:2.658820267242845e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014180374273564667 - rollout_corr/rollout_is_max:2.1670210361480713 - rollout_corr/rollout_is_min:0.20142976939678192 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988207434840839 - rollout_corr/rollout_is_seq_mean:0.9999091625213623 - rollout_corr/rollout_is_seq_std:0.001986477058380842 - rollout_corr/rollout_is_seq_max:1.0120420455932617 - rollout_corr/rollout_is_seq_min:0.9940481185913086 - rollout_corr/rollout_is_seq_max_deviation:0.012042045593261719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1164336544461548) - rollout_corr/training_log_ppl:np.float64(0.10743120299957809) - rollout_corr/kl:np.float64(0.0007887022689292422) - rollout_corr/k3_kl:np.float64(0.0007427555091403804) - rollout_corr/rollout_ppl:np.float64(1.1155504160560668) - rollout_corr/rollout_log_ppl:np.float64(0.1066425001481548) - rollout_corr/log_ppl_diff:np.float64(0.0007887028514232952) - rollout_corr/log_ppl_abs_diff:np.float64(0.001653945950238267) - rollout_corr/log_ppl_diff_max:np.float64(0.012162841856479645) - rollout_corr/log_ppl_diff_min:np.float64(-0.010080903768539429) - rollout_corr/ppl_ratio:np.float64(1.0007913769222796) - rollout_corr/chi2_token:np.float64(0.0014412584714591503) - rollout_corr/chi2_seq:np.float64(0.749183313222602) - actor/pg_loss:np.float64(0.00014212285168468952) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000699050789080502) - actor/ppo_kl:np.float64(5.158489911716302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37614476680755615) - perf/mfu/actor:np.float64(0.06503999971026932) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.41239166259766) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/score/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/score/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0671080518080989) - val-core/sciknoweval/acc/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09981662846229708) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8320624999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04145535225486681) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6367624999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.990625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.030556568705962538) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010087512303978256) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9817375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.040950720392224096) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.990325) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03109914326301299) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9670625000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.050240658088205936) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99635) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01916044898238421) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9449) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05467743751202757) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008915794300729528) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.918325) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.047861241896262015) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0016371701590179605) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00229323236271739 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00229323236271739 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.75 - response_length/max:1682.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.75 - response_length_non_aborted/max:1682.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.746236562728882e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.339398443698883) - timing_s/agent_loop/generate_sequences/max:np.float64(11.241939969360828) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.378861661745759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.241939969360828) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1682 - timing_s/gen:17.086866164579988 - timing_s/reward:0.061425795778632164 - timing_s/old_log_prob:2.430392887443304 - timing_s/adv:0.2020460832864046 - timing_s/update_actor:9.794211328029633 - timing_s/step:29.6614922080189 - timing_s/testing:97.09009108878672 - timing_s/save_checkpoint:7.344661565497518 - timing_s/stop_profile:0.00012345612049102783 - timing_per_token_ms/update_actor:0.054611313052177 - timing_per_token_ms/gen:0.15143634930321173 - timing_per_token_ms/adv:0.001126584013328601 - perf/total_num_tokens:179344 - perf/time_per_step:29.6614922080189 - perf/throughput:755.7947470336425 (TaskRunner pid=2049544) Training Progress: 20%|██ | 20/100 [14:54<1:29:33, 67.17s/it] (TaskRunner pid=2049544) step:21 - global_seqlen/min:20890 - global_seqlen/max:29372 - global_seqlen/minmax_diff:8482 - global_seqlen/balanced_min:23784 - global_seqlen/balanced_max:24194 - global_seqlen/mean:23842.25 - actor/entropy:0.10979560017585754 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417163848876953 - training/rollout_probs_diff_mean:0.0029196166433393955 - training/rollout_probs_diff_std:0.01234018336981535 - training/rollout_actor_probs_pearson_corr:0.9974682927131653 - rollout_corr/rollout_is_mean:1.0000228881835938 - rollout_corr/rollout_is_ratio_fraction_high:1.6375992345274426e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010644395661074668 - rollout_corr/rollout_is_max:2.094970703125 - rollout_corr/rollout_is_min:0.07465586066246033 - rollout_corr/rollout_is_std:0.03303755819797516 - rollout_corr/rollout_is_eff_sample_size:0.9989098287246259 - rollout_corr/rollout_is_seq_mean:0.9999141693115234 - rollout_corr/rollout_is_seq_std:0.0017565247835591435 - rollout_corr/rollout_is_seq_max:1.0051144361495972 - rollout_corr/rollout_is_seq_min:0.9947226643562317 - rollout_corr/rollout_is_seq_max_deviation:0.0052773356437683105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1051466539502144) - rollout_corr/training_log_ppl:np.float64(0.09741823082003975) - rollout_corr/kl:np.float64(0.0006470407322218819) - rollout_corr/k3_kl:np.float64(0.0006291852859376945) - rollout_corr/rollout_ppl:np.float64(1.1044287369586527) - rollout_corr/rollout_log_ppl:np.float64(0.09677118871331913) - rollout_corr/log_ppl_diff:np.float64(0.0006470421067206189) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014577525143977255) - rollout_corr/log_ppl_diff_max:np.float64(0.009566932916641235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005102425813674927) - rollout_corr/ppl_ratio:np.float64(1.0006491616368294) - rollout_corr/chi2_token:np.float64(0.0012274319306015968) - rollout_corr/chi2_seq:np.float64(1.2051882504019886) - actor/pg_loss:np.float64(0.00013444910291582346) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045100462170921674) - actor/ppo_kl:np.float64(-5.319785113400144e-05) - actor/pg_clipfrac_lower:np.float64(2.38095403801708e-05) - actor/grad_norm:np.float64(0.2096654698252678) - perf/mfu/actor:np.float64(0.0669203788543201) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66231536865234) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004104233346879482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004104233346879482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.0703125 - response_length/max:2091.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.0703125 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015165284276008606 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7176804710179567) - timing_s/agent_loop/generate_sequences/max:np.float64(12.88374364003539) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.737000476088724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.88374364003539) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2091 - timing_s/gen:19.051039807498455 - timing_s/reward:0.05670631118118763 - timing_s/old_log_prob:2.6326032262295485 - timing_s/adv:0.14211471565067768 - timing_s/update_actor:10.165599260479212 - timing_s/step:32.148173009976745 - timing_s/stop_profile:0.000135812908411026 - timing_per_token_ms/update_actor:0.053296140572299235 - timing_per_token_ms/gen:0.1559898453082654 - timing_per_token_ms/adv:0.000745078147252659 - perf/total_num_tokens:190738 - perf/time_per_step:32.148173009976745 - perf/throughput:741.6362352100347 (TaskRunner pid=2049544) Training Progress: 21%|██ | 21/100 [15:26<1:14:37, 56.68s/it] (TaskRunner pid=2049544) step:22 - global_seqlen/min:16900 - global_seqlen/max:32996 - global_seqlen/minmax_diff:16096 - global_seqlen/balanced_min:23691 - global_seqlen/balanced_max:23913 - global_seqlen/mean:23732.375 - actor/entropy:0.12188427150249481 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5312178134918213 - training/rollout_probs_diff_mean:0.0033468599431216717 - training/rollout_probs_diff_std:0.013181475922465324 - training/rollout_actor_probs_pearson_corr:0.9972920417785645 - rollout_corr/rollout_is_mean:0.999908983707428 - rollout_corr/rollout_is_ratio_fraction_high:2.4285402105306275e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001942832168424502 - rollout_corr/rollout_is_max:3.412278175354004 - rollout_corr/rollout_is_min:0.14518168568611145 - rollout_corr/rollout_is_std:0.03405214846134186 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:0.999866247177124 - rollout_corr/rollout_is_seq_std:0.00229041394777596 - rollout_corr/rollout_is_seq_max:1.0077131986618042 - rollout_corr/rollout_is_seq_min:0.9842400550842285 - rollout_corr/rollout_is_seq_max_deviation:0.015759944915771484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1243417421355844) - rollout_corr/training_log_ppl:np.float64(0.11367632360634161) - rollout_corr/kl:np.float64(0.0010524871724442164) - rollout_corr/k3_kl:np.float64(0.000926966261786788) - rollout_corr/rollout_ppl:np.float64(1.1231554276309907) - rollout_corr/rollout_log_ppl:np.float64(0.11262383543362375) - rollout_corr/log_ppl_diff:np.float64(0.001052488172717858) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018730249648797326) - rollout_corr/log_ppl_diff_max:np.float64(0.0278562530875206) - rollout_corr/log_ppl_diff_min:np.float64(-0.004908844828605652) - rollout_corr/ppl_ratio:np.float64(1.0010584215633571) - rollout_corr/chi2_token:np.float64(0.0015142448246479034) - rollout_corr/chi2_seq:np.float64(0.5919819474220276) - actor/pg_loss:np.float64(-3.6185141652822495e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007750144300189277) - actor/ppo_kl:np.float64(0.00017040614556229627) - actor/pg_clipfrac_lower:np.float64(5.271592272038106e-06) - actor/grad_norm:np.float64(0.4317408576607704) - perf/mfu/actor:np.float64(0.06862934905562151) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66873407363892) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038765573408454657 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038765573408454657 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:482.54296875 - response_length/max:1955.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:482.54296875 - response_length_non_aborted/max:1955.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013677775859832764 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3632201459258795) - timing_s/agent_loop/generate_sequences/max:np.float64(12.304583990946412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.897737120372767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.304583990946412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1955 - timing_s/gen:18.77624169550836 - timing_s/reward:0.054932184517383575 - timing_s/old_log_prob:2.5277608297765255 - timing_s/adv:0.14852365665137768 - timing_s/update_actor:10.204811798408628 - timing_s/step:31.823464412242174 - timing_s/stop_profile:0.0001289471983909607 - timing_per_token_ms/update_actor:0.05374942351117739 - timing_per_token_ms/gen:0.15199619282211235 - timing_per_token_ms/adv:0.0007822839931284673 - perf/total_num_tokens:189859 - perf/time_per_step:31.823464412242174 - perf/throughput:745.7508300344066 (TaskRunner pid=2049544) Training Progress: 22%|██▏ | 22/100 [15:58<1:04:00, 49.24s/it] (TaskRunner pid=2049544) step:23 - global_seqlen/min:17606 - global_seqlen/max:33708 - global_seqlen/minmax_diff:16102 - global_seqlen/balanced_min:26856 - global_seqlen/balanced_max:26886 - global_seqlen/mean:26877.375 - actor/entropy:0.1653793752193451 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9992972612380981 - training/rollout_probs_diff_mean:0.003593530971556902 - training/rollout_probs_diff_std:0.01221021730452776 - training/rollout_actor_probs_pearson_corr:0.9982529878616333 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:2.7500669602886774e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.18765116087161e-05 - rollout_corr/rollout_is_max:3.486543655395508 - rollout_corr/rollout_is_min:0.00043065936188213527 - rollout_corr/rollout_is_std:0.034376177936792374 - rollout_corr/rollout_is_eff_sample_size:0.9988199109866871 - rollout_corr/rollout_is_seq_mean:1.0000687837600708 - rollout_corr/rollout_is_seq_std:0.0017359366174787283 - rollout_corr/rollout_is_seq_max:1.0058130025863647 - rollout_corr/rollout_is_seq_min:0.9936341047286987 - rollout_corr/rollout_is_seq_max_deviation:0.0063658952713012695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1735385786741972) - rollout_corr/training_log_ppl:np.float64(0.1555803755109082) - rollout_corr/kl:np.float64(0.0008435647158400172) - rollout_corr/k3_kl:np.float64(0.000810335308230492) - rollout_corr/rollout_ppl:np.float64(1.172548221424222) - rollout_corr/rollout_log_ppl:np.float64(0.15473681054572808) - rollout_corr/log_ppl_diff:np.float64(0.0008435649651801214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016919265181059018) - rollout_corr/log_ppl_diff_max:np.float64(0.013718657195568085) - rollout_corr/log_ppl_diff_min:np.float64(-0.005128346383571625) - rollout_corr/ppl_ratio:np.float64(1.000846435315907) - rollout_corr/chi2_token:np.float64(0.001545611536130309) - rollout_corr/chi2_seq:np.float64(0.7387706565205008) - actor/pg_loss:np.float64(-7.304479368031025e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014301339112989808) - actor/ppo_kl:np.float64(0.00022386275868435668) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4195772558450699) - perf/mfu/actor:np.float64(0.07791174639008124) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.34968948364258) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014247066341340542 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014247066341340542 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.16796875 - response_length/max:1852.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.16796875 - response_length_non_aborted/max:1852.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003154464066028595 - timing_s/agent_loop/generate_sequences/min:np.float64(1.197395345196128) - timing_s/agent_loop/generate_sequences/max:np.float64(13.042586963623762) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.787129377022211) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.042586963623762) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1852 - timing_s/gen:18.867352159693837 - timing_s/reward:0.06311722472310066 - timing_s/old_log_prob:2.6495919916778803 - timing_s/adv:0.15099932067096233 - timing_s/update_actor:9.910499073565006 - timing_s/step:31.743287844583392 - timing_s/stop_profile:0.00010893121361732483 - timing_per_token_ms/update_actor:0.046091271346090375 - timing_per_token_ms/gen:0.12971620793046343 - timing_per_token_ms/adv:0.0007022603615074126 - perf/total_num_tokens:215019 - perf/time_per_step:31.743287844583392 - perf/throughput:846.7104961399359 (TaskRunner pid=2049544) Training Progress: 23%|██▎ | 23/100 [16:30<56:36, 44.11s/it] (TaskRunner pid=2049544) step:24 - global_seqlen/min:18041 - global_seqlen/max:30779 - global_seqlen/minmax_diff:12738 - global_seqlen/balanced_min:24022 - global_seqlen/balanced_max:24069 - global_seqlen/mean:24056.125 - actor/entropy:0.143111914396286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8866641521453857 - training/rollout_probs_diff_mean:0.003361048409715295 - training/rollout_probs_diff_std:0.012565950863063335 - training/rollout_actor_probs_pearson_corr:0.9978638887405396 - rollout_corr/rollout_is_mean:1.0000112056732178 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013656703231390566 - rollout_corr/rollout_is_max:1.9401601552963257 - rollout_corr/rollout_is_min:0.01988193951547146 - rollout_corr/rollout_is_std:0.03432759642601013 - rollout_corr/rollout_is_eff_sample_size:0.9988230031268704 - rollout_corr/rollout_is_seq_mean:1.000041127204895 - rollout_corr/rollout_is_seq_std:0.0024783890694379807 - rollout_corr/rollout_is_seq_max:1.0213310718536377 - rollout_corr/rollout_is_seq_min:0.9930093884468079 - rollout_corr/rollout_is_seq_max_deviation:0.021331071853637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.144452238921076) - rollout_corr/training_log_ppl:np.float64(0.13059414884628495) - rollout_corr/kl:np.float64(0.00082983940849779) - rollout_corr/k3_kl:np.float64(0.0008679421140129762) - rollout_corr/rollout_ppl:np.float64(1.1434855586849153) - rollout_corr/rollout_log_ppl:np.float64(0.12976430927665206) - rollout_corr/log_ppl_diff:np.float64(0.0008298395696328953) - rollout_corr/log_ppl_abs_diff:np.float64(0.001792395327356644) - rollout_corr/log_ppl_diff_max:np.float64(0.01518431305885315) - rollout_corr/log_ppl_diff_min:np.float64(-0.016911908984184265) - rollout_corr/ppl_ratio:np.float64(1.000833987724036) - rollout_corr/chi2_token:np.float64(0.0022546271793544292) - rollout_corr/chi2_seq:np.float64(1.743649244075641) - actor/pg_loss:np.float64(0.00021742633543908596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007519598264025262) - actor/ppo_kl:np.float64(9.309058857098762e-05) - actor/pg_clipfrac_lower:np.float64(5.556543328566477e-06) - actor/grad_norm:np.float64(0.3718913719058037) - perf/mfu/actor:np.float64(0.06935155745712235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.33251953125) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010280685499310493 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010280685499310493 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:486.25390625 - response_length/max:1963.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:486.25390625 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013114511966705322 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162827044725418) - timing_s/agent_loop/generate_sequences/max:np.float64(12.901960568502545) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.848180819972185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.901960568502545) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:257 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.82960532233119 - timing_s/reward:0.0641421228647232 - timing_s/old_log_prob:2.5491561722010374 - timing_s/adv:0.1262466385960579 - timing_s/update_actor:10.136537404730916 - timing_s/step:31.79429466277361 - timing_s/stop_profile:0.00012167543172836304 - timing_per_token_ms/update_actor:0.052671291639504056 - timing_per_token_ms/gen:0.15126489442028254 - timing_per_token_ms/adv:0.0006560004915383186 - perf/total_num_tokens:192449 - perf/time_per_step:31.79429466277361 - perf/throughput:756.6176653752331 (TaskRunner pid=2049544) Training Progress: 24%|██▍ | 24/100 [17:02<51:12, 40.43s/it] (TaskRunner pid=2049544) step:25 - global_seqlen/min:17209 - global_seqlen/max:32556 - global_seqlen/minmax_diff:15347 - global_seqlen/balanced_min:25066 - global_seqlen/balanced_max:25097 - global_seqlen/mean:25084.5 - actor/entropy:0.16078905761241913 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7048856616020203 - training/rollout_probs_diff_mean:0.003612553235143423 - training/rollout_probs_diff_std:0.01245187222957611 - training/rollout_actor_probs_pearson_corr:0.9980130791664124 - rollout_corr/rollout_is_mean:1.0001463890075684 - rollout_corr/rollout_is_ratio_fraction_high:3.095304418820888e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.73826104705222e-05 - rollout_corr/rollout_is_max:2.4638538360595703 - rollout_corr/rollout_is_min:0.07782533764839172 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987899420046099 - rollout_corr/rollout_is_seq_mean:1.0001909732818604 - rollout_corr/rollout_is_seq_std:0.0020113997161388397 - rollout_corr/rollout_is_seq_max:1.0118286609649658 - rollout_corr/rollout_is_seq_min:0.9921811819076538 - rollout_corr/rollout_is_seq_max_deviation:0.01182866096496582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1733281780034304) - rollout_corr/training_log_ppl:np.float64(0.15433276955445763) - rollout_corr/kl:np.float64(0.0005177396068951623) - rollout_corr/k3_kl:np.float64(0.0007405701547043009) - rollout_corr/rollout_ppl:np.float64(1.172693099360913) - rollout_corr/rollout_log_ppl:np.float64(0.15381502895615995) - rollout_corr/log_ppl_diff:np.float64(0.0005177405982976779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015419888804899529) - rollout_corr/log_ppl_diff_max:np.float64(0.017087146639823914) - rollout_corr/log_ppl_diff_min:np.float64(-0.010565042495727539) - rollout_corr/ppl_ratio:np.float64(1.0005205699708313) - rollout_corr/chi2_token:np.float64(0.0020056585781276226) - rollout_corr/chi2_seq:np.float64(0.9553266488946974) - actor/pg_loss:np.float64(7.516134064644575e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005029199624004832) - actor/ppo_kl:np.float64(6.852521258160493e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3206869065761566) - perf/mfu/actor:np.float64(0.07269379031279649) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.39129257202148) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005469983443617821 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005469983443617821 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:504.796875 - response_length/max:1635.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:504.796875 - response_length_non_aborted/max:1635.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001018233597278595 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8177736550569534) - timing_s/agent_loop/generate_sequences/max:np.float64(11.67398145236075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.060982199414866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.67398145236075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:335 - timing_s/agent_loop/slowest/response_length:1635 - timing_s/gen:17.524463137611747 - timing_s/reward:0.05791650153696537 - timing_s/old_log_prob:2.436915224418044 - timing_s/adv:0.13883326761424541 - timing_s/update_actor:9.980426739901304 - timing_s/step:30.245057743042707 - timing_s/stop_profile:0.00012283958494663239 - timing_per_token_ms/update_actor:0.049734032669085014 - timing_per_token_ms/gen:0.13560887065970026 - timing_per_token_ms/adv:0.0006918279595678876 - perf/total_num_tokens:200676 - perf/time_per_step:30.245057743042707 - perf/throughput:829.3751730651004 (TaskRunner pid=2049544) Training Progress: 25%|██▌ | 25/100 [17:32<46:43, 37.39s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) step:26 - global_seqlen/min:19588 - global_seqlen/max:26583 - global_seqlen/minmax_diff:6995 - global_seqlen/balanced_min:22329 - global_seqlen/balanced_max:22375 - global_seqlen/mean:22358.5 - actor/entropy:0.14037299156188965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3580157160758972 - training/rollout_probs_diff_mean:0.003176919650286436 - training/rollout_probs_diff_std:0.011454894207417965 - training/rollout_actor_probs_pearson_corr:0.9981624484062195 - rollout_corr/rollout_is_mean:1.0003305673599243 - rollout_corr/rollout_is_ratio_fraction_high:2.6683745090849698e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6683745090849698e-05 - rollout_corr/rollout_is_max:3.044771909713745 - rollout_corr/rollout_is_min:0.3202765882015228 - rollout_corr/rollout_is_std:0.03229307755827904 - rollout_corr/rollout_is_eff_sample_size:0.9989589572900974 - rollout_corr/rollout_is_seq_mean:1.0003631114959717 - rollout_corr/rollout_is_seq_std:0.001997264800593257 - rollout_corr/rollout_is_seq_max:1.009385347366333 - rollout_corr/rollout_is_seq_min:0.9931842684745789 - rollout_corr/rollout_is_seq_max_deviation:0.009385347366333008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.137321101501584) - rollout_corr/training_log_ppl:np.float64(0.12318833442259347) - rollout_corr/kl:np.float64(0.00036465673569097135) - rollout_corr/k3_kl:np.float64(0.0006004141492397252) - rollout_corr/rollout_ppl:np.float64(1.136900127865374) - rollout_corr/rollout_log_ppl:np.float64(0.12282367694933782) - rollout_corr/log_ppl_diff:np.float64(0.00036465747325564735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014688569026475307) - rollout_corr/log_ppl_diff_max:np.float64(0.011015359312295914) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049578845500946045) - rollout_corr/ppl_ratio:np.float64(1.0003668149001896) - rollout_corr/chi2_token:np.float64(0.001733903307467699) - rollout_corr/chi2_seq:np.float64(0.5194167881272733) - actor/pg_loss:np.float64(8.537911344319582e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003864719383273041) - actor/ppo_kl:np.float64(0.00016390427055512635) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1358119323849678) - perf/mfu/actor:np.float64(0.06507624815359347) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.42486190795898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01127944141626358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01127944141626358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.171875 - response_length/max:1781.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.171875 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013301707804203033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8018740545958281) - timing_s/agent_loop/generate_sequences/max:np.float64(11.262868203222752) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332270985694777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.262868203222752) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:17.137579491361976 - timing_s/reward:0.06755838170647621 - timing_s/old_log_prob:2.513957444578409 - timing_s/adv:0.14904913492500782 - timing_s/update_actor:10.074241016060114 - timing_s/step:30.039264272898436 - timing_s/stop_profile:0.00011388026177883148 - timing_per_token_ms/update_actor:0.05632220976396065 - timing_per_token_ms/gen:0.15243159614475021 - timing_per_token_ms/adv:0.0008332912255127122 - perf/total_num_tokens:178868 - perf/time_per_step:30.039264272898436 - perf/throughput:744.309174714773 (TaskRunner pid=2049544) Training Progress: 26%|██▌ | 26/100 [18:02<43:24, 35.20s/it] (TaskRunner pid=2049544) step:27 - global_seqlen/min:20390 - global_seqlen/max:35494 - global_seqlen/minmax_diff:15104 - global_seqlen/balanced_min:27757 - global_seqlen/balanced_max:27828 - global_seqlen/mean:27799.5 - actor/entropy:0.13566820323467255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4101554751396179 - training/rollout_probs_diff_mean:0.002812632592394948 - training/rollout_probs_diff_std:0.010530136525630951 - training/rollout_actor_probs_pearson_corr:0.9983336925506592 - rollout_corr/rollout_is_mean:1.0000394582748413 - rollout_corr/rollout_is_ratio_fraction_high:1.9780045477091335e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.615344005287625e-05 - rollout_corr/rollout_is_max:2.5563464164733887 - rollout_corr/rollout_is_min:0.3321341276168823 - rollout_corr/rollout_is_std:0.03021036833524704 - rollout_corr/rollout_is_eff_sample_size:0.9990882848714958 - rollout_corr/rollout_is_seq_mean:1.0000697374343872 - rollout_corr/rollout_is_seq_std:0.0016781107988208532 - rollout_corr/rollout_is_seq_max:1.0076134204864502 - rollout_corr/rollout_is_seq_min:0.9905708432197571 - rollout_corr/rollout_is_seq_max_deviation:0.00942915678024292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1346074021421373) - rollout_corr/training_log_ppl:np.float64(0.12218492220563348) - rollout_corr/kl:np.float64(0.0005914641850237601) - rollout_corr/k3_kl:np.float64(0.0005361424209464616) - rollout_corr/rollout_ppl:np.float64(1.1339335390366614) - rollout_corr/rollout_log_ppl:np.float64(0.1215934576903237) - rollout_corr/log_ppl_diff:np.float64(0.0005914645153097808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011890121677424759) - rollout_corr/log_ppl_diff_max:np.float64(0.012660838663578033) - rollout_corr/log_ppl_diff_min:np.float64(-0.006112679839134216) - rollout_corr/ppl_ratio:np.float64(1.0005930345505476) - rollout_corr/chi2_token:np.float64(0.0009534833952784538) - rollout_corr/chi2_seq:np.float64(1.2373452726751566) - actor/pg_loss:np.float64(0.00019052473362535238) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004077799751485145) - actor/ppo_kl:np.float64(0.00015187378462222512) - actor/pg_clipfrac_lower:np.float64(2.895663556046202e-06) - actor/grad_norm:np.float64(0.24880316480994225) - perf/mfu/actor:np.float64(0.08016295921139835) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.3461446762085) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011563085718080401 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011563085718080401 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:592.453125 - response_length/max:2410.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:592.453125 - response_length_non_aborted/max:2410.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010118260979652405 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9887365885078907) - timing_s/agent_loop/generate_sequences/max:np.float64(15.905478810891509) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.845138130702253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.905478810891509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:2410 - timing_s/gen:21.729817861691117 - timing_s/reward:0.06810113973915577 - timing_s/old_log_prob:2.5614845752716064 - timing_s/adv:0.1540636345744133 - timing_s/update_actor:10.151232639327645 - timing_s/step:34.75130227394402 - timing_s/stop_profile:0.00013370253145694733 - timing_per_token_ms/update_actor:0.04564485260223945 - timing_per_token_ms/gen:0.1432722648264045 - timing_per_token_ms/adv:0.0006927446292847592 - perf/total_num_tokens:222396 - perf/time_per_step:34.75130227394402 - perf/throughput:799.9556327661317 (TaskRunner pid=2049544) Training Progress: 27%|██▋ | 27/100 [18:37<42:40, 35.08s/it] (TaskRunner pid=2049544) step:28 - global_seqlen/min:17957 - global_seqlen/max:34244 - global_seqlen/minmax_diff:16287 - global_seqlen/balanced_min:25513 - global_seqlen/balanced_max:25550 - global_seqlen/mean:25526.75 - actor/entropy:0.16894282400608063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068703174591064 - training/rollout_probs_diff_mean:0.003456175560131669 - training/rollout_probs_diff_std:0.011879513040184975 - training/rollout_actor_probs_pearson_corr:0.9982811212539673 - rollout_corr/rollout_is_mean:0.9999144077301025 - rollout_corr/rollout_is_ratio_fraction_high:1.4930944416846614e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.972377766738646e-05 - rollout_corr/rollout_is_max:2.6947081089019775 - rollout_corr/rollout_is_min:0.39785826206207275 - rollout_corr/rollout_is_std:0.03413606435060501 - rollout_corr/rollout_is_eff_sample_size:0.9988359665383245 - rollout_corr/rollout_is_seq_mean:0.9999093413352966 - rollout_corr/rollout_is_seq_std:0.002044897060841322 - rollout_corr/rollout_is_seq_max:1.0098328590393066 - rollout_corr/rollout_is_seq_min:0.9918521642684937 - rollout_corr/rollout_is_seq_max_deviation:0.00983285903930664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1787242372520268) - rollout_corr/training_log_ppl:np.float64(0.15832054920610972) - rollout_corr/kl:np.float64(0.0006220183064016283) - rollout_corr/k3_kl:np.float64(0.0006294625401039866) - rollout_corr/rollout_ppl:np.float64(1.1779772732406855) - rollout_corr/rollout_log_ppl:np.float64(0.15769852932862705) - rollout_corr/log_ppl_diff:np.float64(0.0006220198774826713) - rollout_corr/log_ppl_abs_diff:np.float64(0.001586154270626139) - rollout_corr/log_ppl_diff_max:np.float64(0.009446874260902405) - rollout_corr/log_ppl_diff_min:np.float64(-0.006108276546001434) - rollout_corr/ppl_ratio:np.float64(1.0006244094111025) - rollout_corr/chi2_token:np.float64(0.0012534137349575758) - rollout_corr/chi2_seq:np.float64(0.5095949505921453) - actor/pg_loss:np.float64(0.0001528579741716385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005431991780824319) - actor/ppo_kl:np.float64(-0.00011134325540240742) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2734988182783127) - perf/mfu/actor:np.float64(0.07474159440863055) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.44351959228516) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018798992037773132 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018798992037773132 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.2421875 - response_length/max:1995.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.2421875 - response_length_non_aborted/max:1995.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001253858208656311 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8268284611403942) - timing_s/agent_loop/generate_sequences/max:np.float64(13.178542386740446) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.144354823620233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.178542386740446) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1995 - timing_s/gen:19.040658405050635 - timing_s/reward:0.06457959674298763 - timing_s/old_log_prob:2.444152580574155 - timing_s/adv:0.15133203007280827 - timing_s/update_actor:10.063168801367283 - timing_s/step:31.856078814715147 - timing_s/stop_profile:0.00013601407408714294 - timing_per_token_ms/update_actor:0.04927756569758823 - timing_per_token_ms/gen:0.14214750582344632 - timing_per_token_ms/adv:0.0007410463047235168 - perf/total_num_tokens:204214 - perf/time_per_step:31.856078814715147 - perf/throughput:801.3148808574813 (TaskRunner pid=2049544) Training Progress: 28%|██▊ | 28/100 [19:09<40:57, 34.13s/it] (TaskRunner pid=2049544) step:29 - global_seqlen/min:20947 - global_seqlen/max:33732 - global_seqlen/minmax_diff:12785 - global_seqlen/balanced_min:25386 - global_seqlen/balanced_max:25685 - global_seqlen/mean:25470.375 - actor/entropy:0.13968856632709503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5444822311401367 - training/rollout_probs_diff_mean:0.0031616955529898405 - training/rollout_probs_diff_std:0.011410399340093136 - training/rollout_actor_probs_pearson_corr:0.9981111884117126 - rollout_corr/rollout_is_mean:1.000052809715271 - rollout_corr/rollout_is_ratio_fraction_high:7.336810995184351e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.8694487961474806e-05 - rollout_corr/rollout_is_max:3.6677300930023193 - rollout_corr/rollout_is_min:0.22905343770980835 - rollout_corr/rollout_is_std:0.0324532613158226 - rollout_corr/rollout_is_eff_sample_size:0.9989480129782379 - rollout_corr/rollout_is_seq_mean:1.0001238584518433 - rollout_corr/rollout_is_seq_std:0.002332444768399 - rollout_corr/rollout_is_seq_max:1.0094679594039917 - rollout_corr/rollout_is_seq_min:0.9920924305915833 - rollout_corr/rollout_is_seq_max_deviation:0.0094679594039917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.140123389661312) - rollout_corr/training_log_ppl:np.float64(0.12807749381863687) - rollout_corr/kl:np.float64(0.000638476259126719) - rollout_corr/k3_kl:np.float64(0.0006935356665564996) - rollout_corr/rollout_ppl:np.float64(1.1393897044472396) - rollout_corr/rollout_log_ppl:np.float64(0.1274390167054662) - rollout_corr/log_ppl_diff:np.float64(0.0006384771131706657) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017721431377140107) - rollout_corr/log_ppl_diff_max:np.float64(0.011614277958869934) - rollout_corr/log_ppl_diff_min:np.float64(-0.011640861630439758) - rollout_corr/ppl_ratio:np.float64(1.0006417338736355) - rollout_corr/chi2_token:np.float64(0.0015386808663606644) - rollout_corr/chi2_seq:np.float64(0.6557973187882453) - actor/pg_loss:np.float64(8.236418943852186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006260939494495688) - actor/ppo_kl:np.float64(0.00012735309167744013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2651471309363842) - perf/mfu/actor:np.float64(0.07338363198697957) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.38971328735352) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009649740532040596 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009649740532040596 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:532.41796875 - response_length/max:2748.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:532.41796875 - response_length_non_aborted/max:2748.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.792375981807709e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251552730798721) - timing_s/agent_loop/generate_sequences/max:np.float64(16.657433584332466) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.999644991774403) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.657433584332466) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:2748 - timing_s/gen:22.55767191387713 - timing_s/reward:0.06376934424042702 - timing_s/old_log_prob:2.4361399989575148 - timing_s/adv:0.1466422975063324 - timing_s/update_actor:10.275281056761742 - timing_s/step:35.56420364230871 - timing_s/stop_profile:0.00011993944644927979 - timing_per_token_ms/update_actor:0.050427609805321585 - timing_per_token_ms/gen:0.16550137502019185 - timing_per_token_ms/adv:0.000719670879925857 - perf/total_num_tokens:203763 - perf/time_per_step:35.56420364230871 - perf/throughput:716.1801022222058 (TaskRunner pid=2049544) Training Progress: 29%|██▉ | 29/100 [19:45<40:54, 34.57s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2049544) step:30 - global_seqlen/min:16681 - global_seqlen/max:26264 - global_seqlen/minmax_diff:9583 - global_seqlen/balanced_min:22916 - global_seqlen/balanced_max:22938 - global_seqlen/mean:22931.25 - actor/entropy:0.1647506058216095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3170890808105469 - training/rollout_probs_diff_mean:0.0035670390352606773 - training/rollout_probs_diff_std:0.01208495907485485 - training/rollout_actor_probs_pearson_corr:0.9981399178504944 - rollout_corr/rollout_is_mean:0.9999049305915833 - rollout_corr/rollout_is_ratio_fraction_high:8.544525371689815e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00011962335702264681 - rollout_corr/rollout_is_max:2.2645792961120605 - rollout_corr/rollout_is_min:0.1764460951089859 - rollout_corr/rollout_is_std:0.03402237966656685 - rollout_corr/rollout_is_eff_sample_size:0.9988435782394575 - rollout_corr/rollout_is_seq_mean:0.9999919533729553 - rollout_corr/rollout_is_seq_std:0.0021591568365693092 - rollout_corr/rollout_is_seq_max:1.0069876909255981 - rollout_corr/rollout_is_seq_min:0.9930654764175415 - rollout_corr/rollout_is_seq_max_deviation:0.0069876909255981445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1598813771270216) - rollout_corr/training_log_ppl:np.float64(0.14310977509740042) - rollout_corr/kl:np.float64(0.0006737301786081673) - rollout_corr/k3_kl:np.float64(0.0006828237883240718) - rollout_corr/rollout_ppl:np.float64(1.1591023579239845) - rollout_corr/rollout_log_ppl:np.float64(0.14243604317016434) - rollout_corr/log_ppl_diff:np.float64(0.0006737319272360764) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017324861764791422) - rollout_corr/log_ppl_diff_max:np.float64(0.0082569420337677) - rollout_corr/log_ppl_diff_min:np.float64(-0.005524754524230957) - rollout_corr/ppl_ratio:np.float64(1.000676429597661) - rollout_corr/chi2_token:np.float64(0.0013968497514724731) - rollout_corr/chi2_seq:np.float64(0.42728084372356534) - actor/pg_loss:np.float64(2.553162630647421e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004172229162122676) - actor/ppo_kl:np.float64(2.4439394231023925e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29012734070420265) - perf/mfu/actor:np.float64(0.06612954074679292) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.5312614440918) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63125) - val-aux/sciknoweval/reward/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/score/mean@16:np.float64(0.63125) - val-aux/sciknoweval/score/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/score/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06499777900917662) - val-core/sciknoweval/acc/mean@16:np.float64(0.63125) - val-aux/sciknoweval/acc/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09227713650829415) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.81775) - val-core/sciknoweval/acc/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04889979668850037) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6632125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98359375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04653599667599353) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.997425) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.01734892838049699) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9696374999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.05990914656773903) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9836) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.046352514622771394) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00190372690402996) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9472625000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0713750183504594) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9930749999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.029627882526401583) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9146000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07555851957352466) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9977499999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014792280126255261) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8794875000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06529402915117104) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997250000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0034755281407842023) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012676871381700039 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012676871381700039 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.1640625 - response_length/max:1567.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.1640625 - response_length_non_aborted/max:1567.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010706298053264618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7780425660312176) - timing_s/agent_loop/generate_sequences/max:np.float64(10.887974064797163) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.538448243663879) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.887974064797163) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1567 - timing_s/gen:16.679681358858943 - timing_s/reward:0.06917422637343407 - timing_s/old_log_prob:2.485142918303609 - timing_s/adv:0.12575341202318668 - timing_s/update_actor:10.15482790954411 - timing_s/step:29.608009880408645 - timing_s/testing:103.1428439784795 - timing_s/save_checkpoint:6.939728271216154 - timing_s/stop_profile:0.0001257825642824173 - timing_per_token_ms/update_actor:0.05535474466908755 - timing_per_token_ms/gen:0.14251996307789996 - timing_per_token_ms/adv:0.0006854914800936859 - perf/total_num_tokens:183450 - perf/time_per_step:29.608009880408645 - perf/throughput:774.4948104456491 (TaskRunner pid=2049544) Training Progress: 30%|███ | 30/100 [22:05<1:17:09, 66.13s/it] (TaskRunner pid=2049544) step:31 - global_seqlen/min:19206 - global_seqlen/max:40437 - global_seqlen/minmax_diff:21231 - global_seqlen/balanced_min:26988 - global_seqlen/balanced_max:27006 - global_seqlen/mean:26994.75 - actor/entropy:0.15669088065624237 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584149241447449 - training/rollout_probs_diff_mean:0.003061925061047077 - training/rollout_probs_diff_std:0.010589882731437683 - training/rollout_actor_probs_pearson_corr:0.9985439777374268 - rollout_corr/rollout_is_mean:1.0000404119491577 - rollout_corr/rollout_is_ratio_fraction_high:1.3810819837090094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7621639674180187e-05 - rollout_corr/rollout_is_max:2.6365303993225098 - rollout_corr/rollout_is_min:0.22984786331653595 - rollout_corr/rollout_is_std:0.030734580010175705 - rollout_corr/rollout_is_eff_sample_size:0.9990563960283122 - rollout_corr/rollout_is_seq_mean:1.0000174045562744 - rollout_corr/rollout_is_seq_std:0.0015726868296042085 - rollout_corr/rollout_is_seq_max:1.0063960552215576 - rollout_corr/rollout_is_seq_min:0.9952640533447266 - rollout_corr/rollout_is_seq_max_deviation:0.006396055221557617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1623710673302412) - rollout_corr/training_log_ppl:np.float64(0.14397141717927298) - rollout_corr/kl:np.float64(0.0004526557635822215) - rollout_corr/k3_kl:np.float64(0.0005084346583146271) - rollout_corr/rollout_ppl:np.float64(1.1618167613632977) - rollout_corr/rollout_log_ppl:np.float64(0.1435187609968125) - rollout_corr/log_ppl_diff:np.float64(0.0004526561824604869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012202383077237755) - rollout_corr/log_ppl_diff_max:np.float64(0.005235195159912109) - rollout_corr/log_ppl_diff_min:np.float64(-0.004862427711486816) - rollout_corr/ppl_ratio:np.float64(1.0004539461806417) - rollout_corr/chi2_token:np.float64(0.0011370060965418816) - rollout_corr/chi2_seq:np.float64(0.43574777944013476) - actor/pg_loss:np.float64(1.602328848093748e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027079670348939544) - actor/ppo_kl:np.float64(-1.6756775252702028e-05) - actor/pg_clipfrac_lower:np.float64(2.4157391180779086e-06) - actor/grad_norm:np.float64(0.20888379216194153) - perf/mfu/actor:np.float64(0.07792531348505069) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.11697006225586) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006074171047657728 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.006074171047657728 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:565.6796875 - response_length/max:1784.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:565.6796875 - response_length_non_aborted/max:1784.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.553879499435425e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7227725125849247) - timing_s/agent_loop/generate_sequences/max:np.float64(13.10448818653822) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.673034100342193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.10448818653822) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:1784 - timing_s/gen:18.842551900073886 - timing_s/reward:0.07317895255982876 - timing_s/old_log_prob:2.6255607213824987 - timing_s/adv:0.14376060664653778 - timing_s/update_actor:10.008040392771363 - timing_s/step:31.798675939440727 - timing_s/stop_profile:0.00011486746370792389 - timing_per_token_ms/update_actor:0.0463425313846737 - timing_per_token_ms/gen:0.13011554062503547 - timing_per_token_ms/adv:0.000665687803399447 - perf/total_num_tokens:215958 - perf/time_per_step:31.798675939440727 - perf/throughput:848.9268563071744 (TaskRunner pid=2049544) Training Progress: 31%|███ | 31/100 [22:36<1:04:13, 55.85s/it] (TaskRunner pid=2049544) step:32 - global_seqlen/min:15472 - global_seqlen/max:26624 - global_seqlen/minmax_diff:11152 - global_seqlen/balanced_min:21157 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21164.125 - actor/entropy:0.16215530037879944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835716128349304 - training/rollout_probs_diff_mean:0.0033622696064412594 - training/rollout_probs_diff_std:0.011649704538285732 - training/rollout_actor_probs_pearson_corr:0.9983426928520203 - rollout_corr/rollout_is_mean:0.9998896718025208 - rollout_corr/rollout_is_ratio_fraction_high:1.9218758097849786e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.765627429354936e-05 - rollout_corr/rollout_is_max:2.1169991493225098 - rollout_corr/rollout_is_min:0.016281770542263985 - rollout_corr/rollout_is_std:0.033417847007513046 - rollout_corr/rollout_is_eff_sample_size:0.9988841362904098 - rollout_corr/rollout_is_seq_mean:0.9999381303787231 - rollout_corr/rollout_is_seq_std:0.00247986800968647 - rollout_corr/rollout_is_seq_max:1.011742353439331 - rollout_corr/rollout_is_seq_min:0.9914096593856812 - rollout_corr/rollout_is_seq_max_deviation:0.011742353439331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1614093114621937) - rollout_corr/training_log_ppl:np.float64(0.1426902024759329) - rollout_corr/kl:np.float64(0.0009178641532645582) - rollout_corr/k3_kl:np.float64(0.0007639187314083529) - rollout_corr/rollout_ppl:np.float64(1.1602927432395518) - rollout_corr/rollout_log_ppl:np.float64(0.1417723370686872) - rollout_corr/log_ppl_diff:np.float64(0.0009178654072456993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020935917491442524) - rollout_corr/log_ppl_diff_max:np.float64(0.03533336520195007) - rollout_corr/log_ppl_diff_min:np.float64(-0.009082436561584473) - rollout_corr/ppl_ratio:np.float64(1.000924628926441) - rollout_corr/chi2_token:np.float64(0.00115761230699718) - rollout_corr/chi2_seq:np.float64(0.4135602100286633) - actor/pg_loss:np.float64(-7.768673822283745e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003810818443525932) - actor/ppo_kl:np.float64(0.00014974098553555137) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26316603645682335) - perf/mfu/actor:np.float64(0.0603751772224912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.58000946044922) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003006534418091178 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003006534418091178 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.50390625 - response_length/max:1261.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.50390625 - response_length_non_aborted/max:1261.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013849511742591858 - timing_s/agent_loop/generate_sequences/min:np.float64(0.584519986063242) - timing_s/agent_loop/generate_sequences/max:np.float64(9.110312020406127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.030619656012277) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.110312020406127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:1261 - timing_s/gen:14.926933655515313 - timing_s/reward:0.05845559947192669 - timing_s/old_log_prob:2.34299498423934 - timing_s/adv:0.13051477633416653 - timing_s/update_actor:10.037976827472448 - timing_s/step:27.581528935581446 - timing_s/stop_profile:0.00012824125587940216 - timing_per_token_ms/update_actor:0.05928650976282063 - timing_per_token_ms/gen:0.14343855912665462 - timing_per_token_ms/adv:0.000770849115745197 - perf/total_num_tokens:169313 - perf/time_per_step:27.581528935581446 - perf/throughput:767.3296520084244 (TaskRunner pid=2049544) Training Progress: 32%|███▏ | 32/100 [23:04<53:42, 47.39s/it] (TaskRunner pid=2049544) step:33 - global_seqlen/min:21009 - global_seqlen/max:35793 - global_seqlen/minmax_diff:14784 - global_seqlen/balanced_min:26523 - global_seqlen/balanced_max:26619 - global_seqlen/mean:26546.375 - actor/entropy:0.1607956439256668 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141917824745178 - training/rollout_probs_diff_mean:0.0031411326490342617 - training/rollout_probs_diff_std:0.01075077150017023 - training/rollout_actor_probs_pearson_corr:0.998579204082489 - rollout_corr/rollout_is_mean:1.0000613927841187 - rollout_corr/rollout_is_ratio_fraction_high:2.17348788282834e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.0714716053334996e-05 - rollout_corr/rollout_is_max:5.24385929107666 - rollout_corr/rollout_is_min:0.3405860662460327 - rollout_corr/rollout_is_std:0.0315140075981617 - rollout_corr/rollout_is_eff_sample_size:0.9990078527425291 - rollout_corr/rollout_is_seq_mean:1.0000804662704468 - rollout_corr/rollout_is_seq_std:0.0017564821755513549 - rollout_corr/rollout_is_seq_max:1.006698489189148 - rollout_corr/rollout_is_seq_min:0.995316743850708 - rollout_corr/rollout_is_seq_max_deviation:0.006698489189147949 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.176677621435374) - rollout_corr/training_log_ppl:np.float64(0.15545551202376373) - rollout_corr/kl:np.float64(0.000548688222613336) - rollout_corr/k3_kl:np.float64(0.0006223271676191189) - rollout_corr/rollout_ppl:np.float64(1.1760094286873937) - rollout_corr/rollout_log_ppl:np.float64(0.1549068232634454) - rollout_corr/log_ppl_diff:np.float64(0.0005486887603183277) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014138846672722138) - rollout_corr/log_ppl_diff_max:np.float64(0.008264932781457901) - rollout_corr/log_ppl_diff_min:np.float64(-0.005941852927207947) - rollout_corr/ppl_ratio:np.float64(1.0005506034940481) - rollout_corr/chi2_token:np.float64(0.0014625547919422388) - rollout_corr/chi2_seq:np.float64(0.45368791883811355) - actor/pg_loss:np.float64(5.0174305215477943e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003640777423470354) - actor/ppo_kl:np.float64(6.6643967455704e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2803301140666008) - perf/mfu/actor:np.float64(0.07602373904107226) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.56497955322266) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006237910129129887 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006237910129129887 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:539.16796875 - response_length/max:2340.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:539.16796875 - response_length_non_aborted/max:2340.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011204741895198822 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2724983282387257) - timing_s/agent_loop/generate_sequences/max:np.float64(14.548386441543698) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.296186965904781) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.548386441543698) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:2340 - timing_s/gen:20.489616226404905 - timing_s/reward:0.06211276724934578 - timing_s/old_log_prob:2.5394117888063192 - timing_s/adv:0.15707547590136528 - timing_s/update_actor:10.15939660742879 - timing_s/step:33.50294020399451 - timing_s/stop_profile:0.00015977025032043457 - timing_per_token_ms/update_actor:0.04783796567058962 - timing_per_token_ms/gen:0.14844643603356522 - timing_per_token_ms/adv:0.0007396277076501277 - perf/total_num_tokens:212371 - perf/time_per_step:33.50294020399451 - perf/throughput:792.3595612314322 (TaskRunner pid=2049544) Training Progress: 33%|███▎ | 33/100 [23:38<48:17, 43.24s/it] (TaskRunner pid=2049544) step:34 - global_seqlen/min:19210 - global_seqlen/max:32362 - global_seqlen/minmax_diff:13152 - global_seqlen/balanced_min:24672 - global_seqlen/balanced_max:24722 - global_seqlen/mean:24706.625 - actor/entropy:0.1634763479232788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.003025032114237547 - training/rollout_probs_diff_std:0.010739969089627266 - training/rollout_actor_probs_pearson_corr:0.998622715473175 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:2.2830181478639133e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.84905462549068e-05 - rollout_corr/rollout_is_max:3.96623158454895 - rollout_corr/rollout_is_min:0.17613501846790314 - rollout_corr/rollout_is_std:0.03082752786576748 - rollout_corr/rollout_is_eff_sample_size:0.9990505658256099 - rollout_corr/rollout_is_seq_mean:1.0001435279846191 - rollout_corr/rollout_is_seq_std:0.001950009143911302 - rollout_corr/rollout_is_seq_max:1.0084269046783447 - rollout_corr/rollout_is_seq_min:0.9940277338027954 - rollout_corr/rollout_is_seq_max_deviation:0.008426904678344727 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1493624411523342) - rollout_corr/training_log_ppl:np.float64(0.13525041530374438) - rollout_corr/kl:np.float64(0.0005815880843990096) - rollout_corr/k3_kl:np.float64(0.0006954628924091821) - rollout_corr/rollout_ppl:np.float64(1.1486861389130354) - rollout_corr/rollout_log_ppl:np.float64(0.13466882749344222) - rollout_corr/log_ppl_diff:np.float64(0.0005815878103021532) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016140752850333229) - rollout_corr/log_ppl_diff_max:np.float64(0.00731133297085762) - rollout_corr/log_ppl_diff_min:np.float64(-0.007330030202865601) - rollout_corr/ppl_ratio:np.float64(1.0005840898957103) - rollout_corr/chi2_token:np.float64(0.0018483358435332775) - rollout_corr/chi2_seq:np.float64(0.8693230981007218) - actor/pg_loss:np.float64(-7.774424739181995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004309596388338832) - actor/ppo_kl:np.float64(0.00016126566817398214) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24917255342006683) - perf/mfu/actor:np.float64(0.07059737130291072) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.48730087280273) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002149841981008649 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002149841981008649 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:513.30078125 - response_length/max:2220.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:513.30078125 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001704581081867218 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9642345681786537) - timing_s/agent_loop/generate_sequences/max:np.float64(14.358597544953227) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.930665408472123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.358597544953227) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:20.079416058957577 - timing_s/reward:0.055915795266628265 - timing_s/old_log_prob:2.5313415098935366 - timing_s/adv:0.14141708984971046 - timing_s/update_actor:10.167155608534813 - timing_s/step:33.08425388485193 - timing_s/stop_profile:0.0001313425600528717 - timing_per_token_ms/update_actor:0.05143941963205625 - timing_per_token_ms/gen:0.15280557101295672 - timing_per_token_ms/adv:0.000715481626131202 - perf/total_num_tokens:197653 - perf/time_per_step:33.08425388485193 - perf/throughput:746.7789688106661 (TaskRunner pid=2049544) Training Progress: 34%|███▍ | 34/100 [24:11<44:13, 40.21s/it] (TaskRunner pid=2049544) step:35 - global_seqlen/min:18610 - global_seqlen/max:31076 - global_seqlen/minmax_diff:12466 - global_seqlen/balanced_min:24486 - global_seqlen/balanced_max:24516 - global_seqlen/mean:24505.375 - actor/entropy:0.18308240175247192 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351485848426819 - training/rollout_probs_diff_mean:0.003324724268168211 - training/rollout_probs_diff_std:0.011287217028439045 - training/rollout_actor_probs_pearson_corr:0.998609185218811 - rollout_corr/rollout_is_mean:0.9998504519462585 - rollout_corr/rollout_is_ratio_fraction_high:1.5409863408422098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.852466034004465e-05 - rollout_corr/rollout_is_max:3.141425609588623 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.03237971290946007 - rollout_corr/rollout_is_eff_sample_size:0.9989524144661357 - rollout_corr/rollout_is_seq_mean:0.999904215335846 - rollout_corr/rollout_is_seq_std:0.002002937952056527 - rollout_corr/rollout_is_seq_max:1.0084916353225708 - rollout_corr/rollout_is_seq_min:0.9936511516571045 - rollout_corr/rollout_is_seq_max_deviation:0.0084916353225708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1920651025138795) - rollout_corr/training_log_ppl:np.float64(0.16792560920293909) - rollout_corr/kl:np.float64(0.0008154499521539904) - rollout_corr/k3_kl:np.float64(0.0007078099062596266) - rollout_corr/rollout_ppl:np.float64(1.1910648443736136) - rollout_corr/rollout_log_ppl:np.float64(0.1671101573156193) - rollout_corr/log_ppl_diff:np.float64(0.0008154518873197958) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017030747985700145) - rollout_corr/log_ppl_diff_max:np.float64(0.010384440422058105) - rollout_corr/log_ppl_diff_min:np.float64(-0.007758818566799164) - rollout_corr/ppl_ratio:np.float64(1.0008183948229998) - rollout_corr/chi2_token:np.float64(0.0012777894735336304) - rollout_corr/chi2_seq:np.float64(1.3958141808398068) - actor/pg_loss:np.float64(1.1038966476917267e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003179428899784398) - actor/ppo_kl:np.float64(0.00012983981464032013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2326635681092739) - perf/mfu/actor:np.float64(0.07068896275098592) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.57692337036133) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004130806773900986 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004130806773900986 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:506.98046875 - response_length/max:1617.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:506.98046875 - response_length_non_aborted/max:1617.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.149631321430206e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9240005984902382) - timing_s/agent_loop/generate_sequences/max:np.float64(11.936645423993468) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.090721758017025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.936645423993468) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1617 - timing_s/gen:17.894150488078594 - timing_s/reward:0.06287588179111481 - timing_s/old_log_prob:2.4778820667415857 - timing_s/adv:0.1439337357878685 - timing_s/update_actor:10.013244837522507 - timing_s/step:30.680053589865565 - timing_s/stop_profile:0.00010856986045837402 - timing_per_token_ms/update_actor:0.05107677824519369 - timing_per_token_ms/gen:0.13787321140082284 - timing_per_token_ms/adv:0.0007341947215043052 - perf/total_num_tokens:196043 - perf/time_per_step:30.680053589865565 - perf/throughput:798.7396413184485 (TaskRunner pid=2049544) Training Progress: 35%|███▌ | 35/100 [24:41<40:28, 37.37s/it] (TaskRunner pid=2049544) step:36 - global_seqlen/min:24110 - global_seqlen/max:37995 - global_seqlen/minmax_diff:13885 - global_seqlen/balanced_min:31143 - global_seqlen/balanced_max:31201 - global_seqlen/mean:31181.25 - actor/entropy:0.2063475251197815 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35195159912109375 - training/rollout_probs_diff_mean:0.003259577788412571 - training/rollout_probs_diff_std:0.010389592498540878 - training/rollout_actor_probs_pearson_corr:0.9988572597503662 - rollout_corr/rollout_is_mean:0.9999135732650757 - rollout_corr/rollout_is_ratio_fraction_high:5.54268399355351e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.217073597421404e-05 - rollout_corr/rollout_is_max:5.167435169219971 - rollout_corr/rollout_is_min:0.08687802404165268 - rollout_corr/rollout_is_std:0.03102603740990162 - rollout_corr/rollout_is_eff_sample_size:0.9990380727632393 - rollout_corr/rollout_is_seq_mean:0.9999022483825684 - rollout_corr/rollout_is_seq_std:0.0015286069829016924 - rollout_corr/rollout_is_seq_max:1.0051732063293457 - rollout_corr/rollout_is_seq_min:0.9951693415641785 - rollout_corr/rollout_is_seq_max_deviation:0.005173206329345703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2262746798805892) - rollout_corr/training_log_ppl:np.float64(0.19696971234225202) - rollout_corr/kl:np.float64(0.0006086873604012055) - rollout_corr/k3_kl:np.float64(0.0005889998031136656) - rollout_corr/rollout_ppl:np.float64(1.2254916024394333) - rollout_corr/rollout_log_ppl:np.float64(0.19636102448566817) - rollout_corr/log_ppl_diff:np.float64(0.000608687856583856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012519802548922598) - rollout_corr/log_ppl_diff_max:np.float64(0.006760932505130768) - rollout_corr/log_ppl_diff_min:np.float64(-0.005945771932601929) - rollout_corr/ppl_ratio:np.float64(1.0006101776380092) - rollout_corr/chi2_token:np.float64(0.001163150416687131) - rollout_corr/chi2_seq:np.float64(0.9006679092999548) - actor/pg_loss:np.float64(6.59312354400754e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041949365038362885) - actor/ppo_kl:np.float64(-4.479620235997572e-05) - actor/pg_clipfrac_lower:np.float64(3.426535158723709e-06) - actor/grad_norm:np.float64(0.21542762964963913) - perf/mfu/actor:np.float64(0.08866368508784078) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.68338394165039) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03192170336842537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03192170336842537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:704.7578125 - response_length/max:2399.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:704.7578125 - response_length_non_aborted/max:2399.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.613484144210815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1882787812501192) - timing_s/agent_loop/generate_sequences/max:np.float64(16.427959153428674) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.1280951405206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.427959153428674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:2399 - timing_s/gen:22.231742713600397 - timing_s/reward:0.06625362113118172 - timing_s/old_log_prob:2.5154502913355827 - timing_s/adv:0.1720115952193737 - timing_s/update_actor:10.132180897518992 - timing_s/step:35.22234913893044 - timing_s/stop_profile:0.00012091919779777527 - timing_per_token_ms/update_actor:0.04061808337349766 - timing_per_token_ms/gen:0.12322352932412729 - timing_per_token_ms/adv:0.00068956342040238 - perf/total_num_tokens:249450 - perf/time_per_step:35.22234913893044 - perf/throughput:885.2688921175928 (TaskRunner pid=2049544) Training Progress: 36%|███▌ | 36/100 [25:17<39:11, 36.74s/it] (TaskRunner pid=2049544) step:37 - global_seqlen/min:20543 - global_seqlen/max:33018 - global_seqlen/minmax_diff:12475 - global_seqlen/balanced_min:26755 - global_seqlen/balanced_max:26801 - global_seqlen/mean:26782.75 - actor/entropy:0.1873542070388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535153567790985 - training/rollout_probs_diff_mean:0.003138389438390732 - training/rollout_probs_diff_std:0.010506313294172287 - training/rollout_actor_probs_pearson_corr:0.9987807273864746 - rollout_corr/rollout_is_mean:0.9999997019767761 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.388498225831427e-05 - rollout_corr/rollout_is_max:1.984167218208313 - rollout_corr/rollout_is_min:0.38167211413383484 - rollout_corr/rollout_is_std:0.031033720821142197 - rollout_corr/rollout_is_eff_sample_size:0.9990378348031789 - rollout_corr/rollout_is_seq_mean:0.9999757409095764 - rollout_corr/rollout_is_seq_std:0.0020235180854797363 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.991079568862915 - rollout_corr/rollout_is_seq_max_deviation:0.008920431137084961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.17386184213683) - rollout_corr/training_log_ppl:np.float64(0.15399431145488052) - rollout_corr/kl:np.float64(0.000585384931017785) - rollout_corr/k3_kl:np.float64(0.0005648392770467581) - rollout_corr/rollout_ppl:np.float64(1.1731591601856053) - rollout_corr/rollout_log_ppl:np.float64(0.15340892578387866) - rollout_corr/log_ppl_diff:np.float64(0.000585385671001859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015394459333037958) - rollout_corr/log_ppl_diff_max:np.float64(0.009715557098388672) - rollout_corr/log_ppl_diff_min:np.float64(-0.006565690040588379) - rollout_corr/ppl_ratio:np.float64(1.0005877201911062) - rollout_corr/chi2_token:np.float64(0.0010866590309888124) - rollout_corr/chi2_seq:np.float64(1.2919703791849315) - actor/pg_loss:np.float64(-7.207388989627361e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027252964378021716) - actor/ppo_kl:np.float64(2.565641079144143e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2241645734757185) - perf/mfu/actor:np.float64(0.07756524385091812) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.62493515014648) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012340063229203224 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012340063229203224 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:576.3984375 - response_length/max:2563.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:576.3984375 - response_length_non_aborted/max:2563.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.650814950466156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7121520359069109) - timing_s/agent_loop/generate_sequences/max:np.float64(16.06510110758245) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.399860835161235) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.06510110758245) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2563 - timing_s/gen:21.823639852926135 - timing_s/reward:0.07287336513400078 - timing_s/old_log_prob:2.61249627918005 - timing_s/adv:0.16705027595162392 - timing_s/update_actor:10.120526600629091 - timing_s/step:34.892802860587835 - timing_s/stop_profile:0.00011398270726203918 - timing_per_token_ms/update_actor:0.04723435140449119 - timing_per_token_ms/gen:0.14789872357260286 - timing_per_token_ms/adv:0.0007796542361763818 - perf/total_num_tokens:214262 - perf/time_per_step:34.892802860587835 - perf/throughput:767.5723302312205 (TaskRunner pid=2049544) Training Progress: 37%|███▋ | 37/100 [25:52<38:00, 36.20s/it] (TaskRunner pid=2049544) step:38 - global_seqlen/min:20896 - global_seqlen/max:36221 - global_seqlen/minmax_diff:15325 - global_seqlen/balanced_min:26629 - global_seqlen/balanced_max:26693 - global_seqlen/mean:26673.0 - actor/entropy:0.17777901887893677 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311151742935181 - training/rollout_probs_diff_mean:0.003014863934367895 - training/rollout_probs_diff_std:0.01037665270268917 - training/rollout_actor_probs_pearson_corr:0.998752772808075 - rollout_corr/rollout_is_mean:1.0000866651535034 - rollout_corr/rollout_is_ratio_fraction_high:2.75694746960653e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.446184564381838e-05 - rollout_corr/rollout_is_max:4.722990036010742 - rollout_corr/rollout_is_min:0.217980295419693 - rollout_corr/rollout_is_std:0.03060242161154747 - rollout_corr/rollout_is_eff_sample_size:0.9990646060209646 - rollout_corr/rollout_is_seq_mean:1.0000120401382446 - rollout_corr/rollout_is_seq_std:0.0017710411921143532 - rollout_corr/rollout_is_seq_max:1.005569338798523 - rollout_corr/rollout_is_seq_min:0.9887515902519226 - rollout_corr/rollout_is_seq_max_deviation:0.011248409748077393 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1705434750765562) - rollout_corr/training_log_ppl:np.float64(0.15039988774515223) - rollout_corr/kl:np.float64(0.0003758689842143781) - rollout_corr/k3_kl:np.float64(0.0005343400661388387) - rollout_corr/rollout_ppl:np.float64(1.170116110239178) - rollout_corr/rollout_log_ppl:np.float64(0.1500240175955696) - rollout_corr/log_ppl_diff:np.float64(0.00037587014958262444) - rollout_corr/log_ppl_abs_diff:np.float64(0.00122680542699527) - rollout_corr/log_ppl_diff_max:np.float64(0.012495182454586029) - rollout_corr/log_ppl_diff_min:np.float64(-0.005860418081283569) - rollout_corr/ppl_ratio:np.float64(1.000377441989258) - rollout_corr/chi2_token:np.float64(0.0015310256276279688) - rollout_corr/chi2_seq:np.float64(0.8941571062896401) - actor/pg_loss:np.float64(7.285538595169783e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002832129719081422) - actor/ppo_kl:np.float64(-9.715641379060003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20185822248458862) - perf/mfu/actor:np.float64(0.07681988024557596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.53835678100586) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021598960738629103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021598960738629103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:566.75 - response_length/max:2166.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:566.75 - response_length_non_aborted/max:2166.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.976459503173828e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1030525546520948) - timing_s/agent_loop/generate_sequences/max:np.float64(14.58826527185738) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.473840224818559) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.58826527185738) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:328 - timing_s/agent_loop/slowest/response_length:2166 - timing_s/gen:20.538045482710004 - timing_s/reward:0.06479933671653271 - timing_s/old_log_prob:2.464496372267604 - timing_s/adv:0.16802607849240303 - timing_s/update_actor:10.160526245832443 - timing_s/step:33.478492356836796 - timing_s/stop_profile:0.00017846189439296722 - timing_per_token_ms/update_actor:0.04761615793982887 - timing_per_token_ms/gen:0.1415557832674653 - timing_per_token_ms/adv:0.0007874352270667109 - perf/total_num_tokens:213384 - perf/time_per_step:33.478492356836796 - perf/throughput:796.720465058606 (TaskRunner pid=2049544) Training Progress: 38%|███▊ | 38/100 [26:25<36:34, 35.40s/it] (TaskRunner pid=2049544) step:39 - global_seqlen/min:15255 - global_seqlen/max:27450 - global_seqlen/minmax_diff:12195 - global_seqlen/balanced_min:22238 - global_seqlen/balanced_max:22271 - global_seqlen/mean:22258.375 - actor/entropy:0.1585860699415207 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.403981477022171 - training/rollout_probs_diff_mean:0.003201692830771208 - training/rollout_probs_diff_std:0.011194097809493542 - training/rollout_actor_probs_pearson_corr:0.9984230995178223 - rollout_corr/rollout_is_mean:1.0000526905059814 - rollout_corr/rollout_is_ratio_fraction_high:4.465760866878554e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.252065213629976e-05 - rollout_corr/rollout_is_max:3.0618841648101807 - rollout_corr/rollout_is_min:0.25154978036880493 - rollout_corr/rollout_is_std:0.032896533608436584 - rollout_corr/rollout_is_eff_sample_size:0.9989191068729061 - rollout_corr/rollout_is_seq_mean:1.0000567436218262 - rollout_corr/rollout_is_seq_std:0.0022173086181282997 - rollout_corr/rollout_is_seq_max:1.0083491802215576 - rollout_corr/rollout_is_seq_min:0.9902656674385071 - rollout_corr/rollout_is_seq_max_deviation:0.00973433256149292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1577396178618073) - rollout_corr/training_log_ppl:np.float64(0.13937123074720148) - rollout_corr/kl:np.float64(0.00047606502702546294) - rollout_corr/k3_kl:np.float64(0.0006253823398196801) - rollout_corr/rollout_ppl:np.float64(1.157200226560235) - rollout_corr/rollout_log_ppl:np.float64(0.1388951658955193) - rollout_corr/log_ppl_diff:np.float64(0.0004760648516821675) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017094570139306597) - rollout_corr/log_ppl_diff_max:np.float64(0.008976280689239502) - rollout_corr/log_ppl_diff_min:np.float64(-0.007652416825294495) - rollout_corr/ppl_ratio:np.float64(1.0004788676742464) - rollout_corr/chi2_token:np.float64(0.001593738328665495) - rollout_corr/chi2_seq:np.float64(1.153579653473571) - actor/pg_loss:np.float64(-4.086887929588556e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021084232048451668) - actor/ppo_kl:np.float64(-5.276580297675082e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22345420345664024) - perf/mfu/actor:np.float64(0.06472844108846296) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.6031494140625) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0039901575073599815 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0039901575073599815 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:437.35546875 - response_length/max:1579.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.35546875 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015556812286376953 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7400109507143497) - timing_s/agent_loop/generate_sequences/max:np.float64(10.687160091474652) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.344982730093761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.687160091474652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:300 - timing_s/agent_loop/slowest/response_length:1564 - timing_s/gen:16.393727527931333 - timing_s/reward:0.060859134420752525 - timing_s/old_log_prob:2.4244355112314224 - timing_s/adv:0.15019422583281994 - timing_s/update_actor:9.942341027781367 - timing_s/step:29.05965017527342 - timing_s/stop_profile:0.00014510564506053925 - timing_per_token_ms/update_actor:0.05583483198897812 - timing_per_token_ms/gen:0.14642093841654236 - timing_per_token_ms/adv:0.0008434702995660057 - perf/total_num_tokens:178067 - perf/time_per_step:29.05965017527342 - perf/throughput:765.954678247966 (TaskRunner pid=2049544) Training Progress: 39%|███▉ | 39/100 [26:54<34:04, 33.52s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2049544) step:40 - global_seqlen/min:20667 - global_seqlen/max:43425 - global_seqlen/minmax_diff:22758 - global_seqlen/balanced_min:28994 - global_seqlen/balanced_max:29096 - global_seqlen/mean:29057.375 - actor/entropy:0.16825498640537262 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38080501556396484 - training/rollout_probs_diff_mean:0.002904911758378148 - training/rollout_probs_diff_std:0.010255875997245312 - training/rollout_actor_probs_pearson_corr:0.9987383484840393 - rollout_corr/rollout_is_mean:0.9999452829360962 - rollout_corr/rollout_is_ratio_fraction_high:1.811298898246605e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.62259779649321e-05 - rollout_corr/rollout_is_max:2.355332374572754 - rollout_corr/rollout_is_min:0.382403701543808 - rollout_corr/rollout_is_std:0.029906976968050003 - rollout_corr/rollout_is_eff_sample_size:0.9991061339931943 - rollout_corr/rollout_is_seq_mean:0.9999656081199646 - rollout_corr/rollout_is_seq_std:0.0016488394467160106 - rollout_corr/rollout_is_seq_max:1.009485125541687 - rollout_corr/rollout_is_seq_min:0.9944038391113281 - rollout_corr/rollout_is_seq_max_deviation:0.009485125541687012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.157492823433131) - rollout_corr/training_log_ppl:np.float64(0.14261198020540178) - rollout_corr/kl:np.float64(0.0005399936895650326) - rollout_corr/k3_kl:np.float64(0.0005378889725164981) - rollout_corr/rollout_ppl:np.float64(1.156866628676653) - rollout_corr/rollout_log_ppl:np.float64(0.1420719847228611) - rollout_corr/log_ppl_diff:np.float64(0.0005399954825406894) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012537419388536364) - rollout_corr/log_ppl_diff_max:np.float64(0.009929120540618896) - rollout_corr/log_ppl_diff_min:np.float64(-0.007549874484539032) - rollout_corr/ppl_ratio:np.float64(1.0005416304338723) - rollout_corr/chi2_token:np.float64(0.0010685438755899668) - rollout_corr/chi2_seq:np.float64(1.119046370498836) - actor/pg_loss:np.float64(-2.1900166757404804e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024728815151320305) - actor/ppo_kl:np.float64(-7.47417178104115e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1894925720989704) - perf/mfu/actor:np.float64(0.08261510768114952) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.61338424682617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/score/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/score/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07283536359808326) - val-core/sciknoweval/acc/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0950390140033621) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8010999999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04192728514733711) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.654725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08980726114743592) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9929124999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.038290550480143085) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9296875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11086911079562345) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9614625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08902370132140851) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996375000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0060893990521829235) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8815249999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12288515049099524) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9828125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.060772813601514916) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8237249999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11143746284761198) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9951625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03001908595873743) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7747124999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0752713841460063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.008297615319292063) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004764470737427473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004764470737427473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.98046875 - response_length/max:3478.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.98046875 - response_length_non_aborted/max:3478.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.821169078350067e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9843795355409384) - timing_s/agent_loop/generate_sequences/max:np.float64(20.97976647876203) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.125959783450526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.97976647876203) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:3478 - timing_s/gen:26.76995438337326 - timing_s/reward:0.06942468881607056 - timing_s/old_log_prob:2.522913720458746 - timing_s/adv:0.12163343653082848 - timing_s/update_actor:10.376626841723919 - timing_s/step:39.94687524065375 - timing_s/testing:129.93085425533354 - timing_s/save_checkpoint:6.843135168775916 - timing_s/stop_profile:9.070523083209991e-05 - timing_per_token_ms/update_actor:0.044638524822544705 - timing_per_token_ms/gen:0.1616279615242277 - timing_per_token_ms/adv:0.0005232468372092648 - perf/total_num_tokens:232459 - perf/time_per_step:39.94687524065375 - perf/throughput:727.4004493454958 (TaskRunner pid=2049544) Training Progress: 40%|████ | 40/100 [29:51<1:16:29, 76.50s/it] (TaskRunner pid=2049544) step:41 - global_seqlen/min:24357 - global_seqlen/max:35343 - global_seqlen/minmax_diff:10986 - global_seqlen/balanced_min:29480 - global_seqlen/balanced_max:29507 - global_seqlen/mean:29501.125 - actor/entropy:0.1756914258003235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44652730226516724 - training/rollout_probs_diff_mean:0.00312806130386889 - training/rollout_probs_diff_std:0.010616622865200043 - training/rollout_actor_probs_pearson_corr:0.9987709522247314 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:1.81861159944674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.424815647827927e-05 - rollout_corr/rollout_is_max:3.465665340423584 - rollout_corr/rollout_is_min:0.21781140565872192 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990134444982739 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014667019713670015 - rollout_corr/rollout_is_seq_max:1.0048595666885376 - rollout_corr/rollout_is_seq_min:0.9947068691253662 - rollout_corr/rollout_is_seq_max_deviation:0.005293130874633789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1796441837213933) - rollout_corr/training_log_ppl:np.float64(0.16063264349941164) - rollout_corr/kl:np.float64(0.0006431586379722631) - rollout_corr/k3_kl:np.float64(0.0005324390123746525) - rollout_corr/rollout_ppl:np.float64(1.178866429720074) - rollout_corr/rollout_log_ppl:np.float64(0.15998948468040908) - rollout_corr/log_ppl_diff:np.float64(0.000643158819002565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012713416290353052) - rollout_corr/log_ppl_diff_max:np.float64(0.007460832595825195) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031778663396835327) - rollout_corr/ppl_ratio:np.float64(1.0006445937324315) - rollout_corr/chi2_token:np.float64(0.0008295027073472738) - rollout_corr/chi2_seq:np.float64(0.9563508839346468) - actor/pg_loss:np.float64(9.131035767495632e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002985205437653349) - actor/ppo_kl:np.float64(3.691702790131757e-05) - actor/pg_clipfrac_lower:np.float64(5.99118084210204e-06) - actor/grad_norm:np.float64(0.2258460782468319) - perf/mfu/actor:np.float64(0.08484757615781298) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.7967414855957) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01819217950105667 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01819217950105667 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.37890625 - response_length/max:1915.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.37890625 - response_length_non_aborted/max:1915.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.878321826457977e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2621874269098043) - timing_s/agent_loop/generate_sequences/max:np.float64(14.428985564038157) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.6339426440390525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.428985564038157) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1915 - timing_s/gen:20.114043543115258 - timing_s/reward:0.06382922269403934 - timing_s/old_log_prob:2.600179996341467 - timing_s/adv:0.1322335172444582 - timing_s/update_actor:9.928239937871695 - timing_s/step:32.94103864394128 - timing_s/stop_profile:0.00011893920600414276 - timing_per_token_ms/update_actor:0.04206720903809471 - timing_per_token_ms/gen:0.12193211451867568 - timing_per_token_ms/adv:0.0005602901467505824 - perf/total_num_tokens:236009 - perf/time_per_step:32.94103864394128 - perf/throughput:895.5736131722133 (TaskRunner pid=2049544) Training Progress: 41%|████ | 41/100 [30:24<1:02:23, 63.45s/it] (TaskRunner pid=2049544) step:42 - global_seqlen/min:25542 - global_seqlen/max:34694 - global_seqlen/minmax_diff:9152 - global_seqlen/balanced_min:30320 - global_seqlen/balanced_max:30380 - global_seqlen/mean:30366.25 - actor/entropy:0.19629985094070435 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31755661964416504 - training/rollout_probs_diff_mean:0.0030814865604043007 - training/rollout_probs_diff_std:0.010210192762315273 - training/rollout_actor_probs_pearson_corr:0.9988877177238464 - rollout_corr/rollout_is_mean:0.9999521374702454 - rollout_corr/rollout_is_ratio_fraction_high:1.7170919818454422e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7170919818454422e-05 - rollout_corr/rollout_is_max:2.3355937004089355 - rollout_corr/rollout_is_min:0.20813822746276855 - rollout_corr/rollout_is_std:0.03025965206325054 - rollout_corr/rollout_is_eff_sample_size:0.9990850720973192 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0012307165889069438 - rollout_corr/rollout_is_seq_max:1.006462574005127 - rollout_corr/rollout_is_seq_min:0.9960255026817322 - rollout_corr/rollout_is_seq_max_deviation:0.006462574005126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1838456057012081) - rollout_corr/training_log_ppl:np.float64(0.161605034765671) - rollout_corr/kl:np.float64(0.00040539363182290344) - rollout_corr/k3_kl:np.float64(0.0005141745206458381) - rollout_corr/rollout_ppl:np.float64(1.1833340604789555) - rollout_corr/rollout_log_ppl:np.float64(0.1611996397114126) - rollout_corr/log_ppl_diff:np.float64(0.00040539505425840616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010806201898958534) - rollout_corr/log_ppl_diff_max:np.float64(0.005684956908226013) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034256726503372192) - rollout_corr/ppl_ratio:np.float64(1.000406396575272) - rollout_corr/chi2_token:np.float64(0.0012630913406610489) - rollout_corr/chi2_seq:np.float64(1.0963045465759933) - actor/pg_loss:np.float64(-1.263362355530262e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005835516549268505) - actor/ppo_kl:np.float64(-5.1322627289351885e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21340569853782654) - perf/mfu/actor:np.float64(0.0874389401304758) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24371719360352) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002196446730522439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002196446730522439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:682.4765625 - response_length/max:2365.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:682.4765625 - response_length_non_aborted/max:2365.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014196522533893585 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5289256758987904) - timing_s/agent_loop/generate_sequences/max:np.float64(16.370171174407005) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.762006698663754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.370171174407005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:2365 - timing_s/gen:22.206629561260343 - timing_s/reward:0.06503966823220253 - timing_s/old_log_prob:2.5655108485370874 - timing_s/adv:0.14332416281104088 - timing_s/update_actor:10.220189092680812 - timing_s/step:35.30416659079492 - timing_s/stop_profile:0.00013269484043121338 - timing_per_token_ms/update_actor:0.04207051040497597 - timing_per_token_ms/gen:0.12710274827008908 - timing_per_token_ms/adv:0.000589981323060309 - perf/total_num_tokens:242930 - perf/time_per_step:35.30416659079492 - perf/throughput:860.1321864348324 (TaskRunner pid=2049544) Training Progress: 42%|████▏ | 42/100 [31:00<53:11, 55.03s/it] (TaskRunner pid=2049544) step:43 - global_seqlen/min:22214 - global_seqlen/max:34661 - global_seqlen/minmax_diff:12447 - global_seqlen/balanced_min:28283 - global_seqlen/balanced_max:28875 - global_seqlen/mean:28428.25 - actor/entropy:0.17894256114959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9968691468238831 - training/rollout_probs_diff_mean:0.0030561003368347883 - training/rollout_probs_diff_std:0.010964183136820793 - training/rollout_actor_probs_pearson_corr:0.9986260533332825 - rollout_corr/rollout_is_mean:0.9999217391014099 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6834687711670995e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:3.1283660064218566e-05 - rollout_corr/rollout_is_std:0.02998857945203781 - rollout_corr/rollout_is_eff_sample_size:0.9991012551699266 - rollout_corr/rollout_is_seq_mean:0.9999701976776123 - rollout_corr/rollout_is_seq_std:0.0015368059976026416 - rollout_corr/rollout_is_seq_max:1.007220983505249 - rollout_corr/rollout_is_seq_min:0.9942285418510437 - rollout_corr/rollout_is_seq_max_deviation:0.0072209835052490234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1693742866627872) - rollout_corr/training_log_ppl:np.float64(0.15083149004203733) - rollout_corr/kl:np.float64(0.0007233850172774225) - rollout_corr/k3_kl:np.float64(0.0006608226789381888) - rollout_corr/rollout_ppl:np.float64(1.168502108193934) - rollout_corr/rollout_log_ppl:np.float64(0.15010810326202773) - rollout_corr/log_ppl_diff:np.float64(0.000723386780009605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408983988687396) - rollout_corr/log_ppl_diff_max:np.float64(0.025284215807914734) - rollout_corr/log_ppl_diff_min:np.float64(-0.005987722426652908) - rollout_corr/ppl_ratio:np.float64(1.0007265941239893) - rollout_corr/chi2_token:np.float64(0.0009258103091269732) - rollout_corr/chi2_seq:np.float64(1.824508806457743) - actor/pg_loss:np.float64(0.0001151182223111391) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000508402825516896) - actor/ppo_kl:np.float64(5.8405713696974004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2429964765906334) - perf/mfu/actor:np.float64(0.08157217656976501) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.25969696044922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018738869577646255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018738869577646255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:618.5703125 - response_length/max:2843.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:618.5703125 - response_length_non_aborted/max:2843.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.98380172252655e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5165256280452013) - timing_s/agent_loop/generate_sequences/max:np.float64(18.731039391830564) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.20822462200158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.731039391830564) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2843 - timing_s/gen:24.405709667131305 - timing_s/reward:0.0670168325304985 - timing_s/old_log_prob:2.534782351925969 - timing_s/adv:0.14802170917391777 - timing_s/update_actor:10.064561607316136 - timing_s/step:37.307200910523534 - timing_s/stop_profile:0.00010807998478412628 - timing_per_token_ms/update_actor:0.044254226022161655 - timing_per_token_ms/gen:0.15412120734008175 - timing_per_token_ms/adv:0.0006508565826858748 - perf/total_num_tokens:227426 - perf/time_per_step:37.307200910523534 - perf/throughput:762.0043666149454 (TaskRunner pid=2049544) Training Progress: 43%|████▎ | 43/100 [31:37<47:14, 49.73s/it] (TaskRunner pid=2049544) step:44 - global_seqlen/min:22462 - global_seqlen/max:40198 - global_seqlen/minmax_diff:17736 - global_seqlen/balanced_min:31295 - global_seqlen/balanced_max:31374 - global_seqlen/mean:31349.5 - actor/entropy:0.17300812900066376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6893051266670227 - training/rollout_probs_diff_mean:0.002969349967315793 - training/rollout_probs_diff_std:0.010275507345795631 - training/rollout_actor_probs_pearson_corr:0.9987791776657104 - rollout_corr/rollout_is_mean:0.9999350309371948 - rollout_corr/rollout_is_ratio_fraction_high:1.112074914999539e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.448299659998156e-05 - rollout_corr/rollout_is_max:2.314483165740967 - rollout_corr/rollout_is_min:0.03194784000515938 - rollout_corr/rollout_is_std:0.030064012855291367 - rollout_corr/rollout_is_eff_sample_size:0.9990967333762735 - rollout_corr/rollout_is_seq_mean:0.9999725222587585 - rollout_corr/rollout_is_seq_std:0.0014090986223891377 - rollout_corr/rollout_is_seq_max:1.0068600177764893 - rollout_corr/rollout_is_seq_min:0.9924980998039246 - rollout_corr/rollout_is_seq_max_deviation:0.0075019001960754395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1756953983567655) - rollout_corr/training_log_ppl:np.float64(0.15532817154598888) - rollout_corr/kl:np.float64(0.00048252152897987344) - rollout_corr/k3_kl:np.float64(0.0005270084845747647) - rollout_corr/rollout_ppl:np.float64(1.1751229297369719) - rollout_corr/rollout_log_ppl:np.float64(0.15484564797225175) - rollout_corr/log_ppl_diff:np.float64(0.00048252357373712584) - rollout_corr/log_ppl_abs_diff:np.float64(0.00115150140482001) - rollout_corr/log_ppl_diff_max:np.float64(0.01116035133600235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005915746092796326) - rollout_corr/ppl_ratio:np.float64(1.0004838448949158) - rollout_corr/chi2_token:np.float64(0.0010843609925359488) - rollout_corr/chi2_seq:np.float64(0.9741545785218477) - actor/pg_loss:np.float64(-0.00012030208017677069) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002879910933870633) - actor/ppo_kl:np.float64(-3.409391269992845e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15622974559664726) - perf/mfu/actor:np.float64(0.08872996761202245) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24708938598633) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017787639051675797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017787639051675797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:702.515625 - response_length/max:2613.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:702.515625 - response_length_non_aborted/max:2613.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001371428370475769 - timing_s/agent_loop/generate_sequences/min:np.float64(2.046190181747079) - timing_s/agent_loop/generate_sequences/max:np.float64(18.486025908961892) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.959546060519642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.486025908961892) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:2613 - timing_s/gen:24.597592102363706 - timing_s/reward:0.07184095494449139 - timing_s/old_log_prob:2.5473712403327227 - timing_s/adv:0.13307522423565388 - timing_s/update_actor:10.182054167613387 - timing_s/step:37.62097364850342 - timing_s/stop_profile:0.0001139417290687561 - timing_per_token_ms/update_actor:0.040598949614879774 - timing_per_token_ms/gen:0.13677182503927685 - timing_per_token_ms/adv:0.0005306114301490211 - perf/total_num_tokens:250796 - perf/time_per_step:37.62097364850342 - perf/throughput:833.2984758156863 (TaskRunner pid=2049544) Training Progress: 44%|████▍ | 44/100 [32:15<43:02, 46.11s/it] (TaskRunner pid=2049544) step:45 - global_seqlen/min:20785 - global_seqlen/max:36095 - global_seqlen/minmax_diff:15310 - global_seqlen/balanced_min:29569 - global_seqlen/balanced_max:29783 - global_seqlen/mean:29630.5 - actor/entropy:0.2007351815700531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102352559566498 - training/rollout_probs_diff_mean:0.0033135332632809877 - training/rollout_probs_diff_std:0.01088031567633152 - training/rollout_actor_probs_pearson_corr:0.998772919178009 - rollout_corr/rollout_is_mean:1.0001227855682373 - rollout_corr/rollout_is_ratio_fraction_high:6.0338379626045935e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.033837780705653e-05 - rollout_corr/rollout_is_max:5.347679138183594 - rollout_corr/rollout_is_min:0.0047262562438845634 - rollout_corr/rollout_is_std:0.03222286328673363 - rollout_corr/rollout_is_eff_sample_size:0.998963001987778 - rollout_corr/rollout_is_seq_mean:1.00006902217865 - rollout_corr/rollout_is_seq_std:0.001625648234039545 - rollout_corr/rollout_is_seq_max:1.0073453187942505 - rollout_corr/rollout_is_seq_min:0.9958909749984741 - rollout_corr/rollout_is_seq_max_deviation:0.007345318794250488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.208143400028348) - rollout_corr/training_log_ppl:np.float64(0.17995586975303013) - rollout_corr/kl:np.float64(0.0005010489769254889) - rollout_corr/k3_kl:np.float64(0.0006270660993550337) - rollout_corr/rollout_ppl:np.float64(1.20752296410501) - rollout_corr/rollout_log_ppl:np.float64(0.1794548208345077) - rollout_corr/log_ppl_diff:np.float64(0.0005010489185224287) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013183121991460212) - rollout_corr/log_ppl_diff_max:np.float64(0.006140023469924927) - rollout_corr/log_ppl_diff_min:np.float64(-0.006050072610378265) - rollout_corr/ppl_ratio:np.float64(1.000502604758367) - rollout_corr/chi2_token:np.float64(0.0014851994346827269) - rollout_corr/chi2_seq:np.float64(1.3357745304238051) - actor/pg_loss:np.float64(-6.521574687212706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033872173025883967) - actor/ppo_kl:np.float64(-1.556063463681312e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20266040414571762) - perf/mfu/actor:np.float64(0.08389738845778953) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0823974609375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005806060507893562 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005806060507893562 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:647.390625 - response_length/max:2551.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:647.390625 - response_length_non_aborted/max:2551.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00031680427491664886 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9299852326512337) - timing_s/agent_loop/generate_sequences/max:np.float64(16.61781131848693) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.397257668249949) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.61781131848693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:324 - timing_s/agent_loop/slowest/response_length:2551 - timing_s/gen:22.67037677206099 - timing_s/reward:0.06372027657926083 - timing_s/old_log_prob:2.7545996010303497 - timing_s/adv:0.11587526835501194 - timing_s/update_actor:10.353036934509873 - timing_s/step:36.06465018354356 - timing_s/stop_profile:0.00011578015983104706 - timing_per_token_ms/update_actor:0.04367559159696037 - timing_per_token_ms/gen:0.13678937544988892 - timing_per_token_ms/adv:0.0004888344288613588 - perf/total_num_tokens:237044 - perf/time_per_step:36.06465018354356 - perf/throughput:821.5939943740397 (TaskRunner pid=2049544) Training Progress: 45%|████▌ | 45/100 [32:51<39:37, 43.23s/it] (TaskRunner pid=2049544) step:46 - global_seqlen/min:16300 - global_seqlen/max:40451 - global_seqlen/minmax_diff:24151 - global_seqlen/balanced_min:26008 - global_seqlen/balanced_max:26038 - global_seqlen/mean:26025.75 - actor/entropy:0.16465333104133606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42921197414398193 - training/rollout_probs_diff_mean:0.0029608916956931353 - training/rollout_probs_diff_std:0.01070943009108305 - training/rollout_actor_probs_pearson_corr:0.9986132979393005 - rollout_corr/rollout_is_mean:0.9999878406524658 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.257039058255032e-05 - rollout_corr/rollout_is_max:1.9571115970611572 - rollout_corr/rollout_is_min:0.060181617736816406 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990728161485534 - rollout_corr/rollout_is_seq_mean:1.000013828277588 - rollout_corr/rollout_is_seq_std:0.0019006438087671995 - rollout_corr/rollout_is_seq_max:1.008240818977356 - rollout_corr/rollout_is_seq_min:0.993937611579895 - rollout_corr/rollout_is_seq_max_deviation:0.008240818977355957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.153148335404694) - rollout_corr/training_log_ppl:np.float64(0.136645665843389) - rollout_corr/kl:np.float64(0.0006094530078737392) - rollout_corr/k3_kl:np.float64(0.0006117465548527434) - rollout_corr/rollout_ppl:np.float64(1.152437952812761) - rollout_corr/rollout_log_ppl:np.float64(0.13603621201764327) - rollout_corr/log_ppl_diff:np.float64(0.0006094538257457316) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014239892771001905) - rollout_corr/log_ppl_diff_max:np.float64(0.014275498688220978) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409347057342529) - rollout_corr/ppl_ratio:np.float64(1.0006118323653936) - rollout_corr/chi2_token:np.float64(0.0012153126299381256) - rollout_corr/chi2_seq:np.float64(0.35479475394822657) - actor/pg_loss:np.float64(-5.846214480698109e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001732110890770855) - actor/ppo_kl:np.float64(5.357556297269639e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1466388632543385) - perf/mfu/actor:np.float64(0.07518990372656345) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.19413375854492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023664643988013268 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023664643988013268 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:561.8671875 - response_length/max:2403.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:561.8671875 - response_length_non_aborted/max:2403.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002136826515197754 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7386901136487722) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293935690075159) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.345510819919582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293935690075159) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2403 - timing_s/gen:21.075742207467556 - timing_s/reward:0.06618223898112774 - timing_s/old_log_prob:2.503137046471238 - timing_s/adv:0.11277000606060028 - timing_s/update_actor:10.213601699098945 - timing_s/step:34.05827110260725 - timing_s/stop_profile:0.00012739188969135284 - timing_per_token_ms/update_actor:0.04905527073714948 - timing_per_token_ms/gen:0.14652416056582793 - timing_per_token_ms/adv:0.000541627071557017 - perf/total_num_tokens:208206 - perf/time_per_step:34.05827110260725 - perf/throughput:764.1535861169318 (TaskRunner pid=2049544) Training Progress: 46%|████▌ | 46/100 [33:25<36:26, 40.50s/it] (TaskRunner pid=2049544) step:47 - global_seqlen/min:23716 - global_seqlen/max:42058 - global_seqlen/minmax_diff:18342 - global_seqlen/balanced_min:31374 - global_seqlen/balanced_max:31661 - global_seqlen/mean:31456.375 - actor/entropy:0.14881354570388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4682484567165375 - training/rollout_probs_diff_mean:0.002767270663753152 - training/rollout_probs_diff_std:0.010147105902433395 - training/rollout_actor_probs_pearson_corr:0.9986124634742737 - rollout_corr/rollout_is_mean:1.0000232458114624 - rollout_corr/rollout_is_ratio_fraction_high:5.438477273855824e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.350781819084659e-05 - rollout_corr/rollout_is_max:2.263822555541992 - rollout_corr/rollout_is_min:0.43962690234184265 - rollout_corr/rollout_is_std:0.02890361100435257 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0001194477081299 - rollout_corr/rollout_is_seq_std:0.0015050210058689117 - rollout_corr/rollout_is_seq_max:1.005627155303955 - rollout_corr/rollout_is_seq_min:0.993998646736145 - rollout_corr/rollout_is_seq_max_deviation:0.0060013532638549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1467792866751552) - rollout_corr/training_log_ppl:np.float64(0.13201592379482463) - rollout_corr/kl:np.float64(0.0004759397659170883) - rollout_corr/k3_kl:np.float64(0.0005555009257420807) - rollout_corr/rollout_ppl:np.float64(1.146229465957731) - rollout_corr/rollout_log_ppl:np.float64(0.13153998326015426) - rollout_corr/log_ppl_diff:np.float64(0.00047594053467037156) - rollout_corr/log_ppl_abs_diff:np.float64(0.001261575052922126) - rollout_corr/log_ppl_diff_max:np.float64(0.008257180452346802) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055009350180625916) - rollout_corr/ppl_ratio:np.float64(1.000477674184367) - rollout_corr/chi2_token:np.float64(0.0013230419717729092) - rollout_corr/chi2_seq:np.float64(0.636790500022471) - actor/pg_loss:np.float64(-0.00010656809899955988) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031795622612662555) - actor/ppo_kl:np.float64(0.00011273976376457995) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1950441598892212) - perf/mfu/actor:np.float64(0.0887384677499009) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.28681945800781) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011680489405989647 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011680489405989647 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:718.26171875 - response_length/max:4213.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:718.26171875 - response_length_non_aborted/max:4213.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017112679779529572 - timing_s/agent_loop/generate_sequences/min:np.float64(0.828915087506175) - timing_s/agent_loop/generate_sequences/max:np.float64(25.74352060444653) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.723139668560179) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.74352060444653) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:4213 - timing_s/gen:31.583294926211238 - timing_s/reward:0.0717991292476654 - timing_s/old_log_prob:2.5992581117898226 - timing_s/adv:0.12912281975150108 - timing_s/update_actor:10.13827026821673 - timing_s/step:44.60779097676277 - timing_s/stop_profile:0.00012019835412502289 - timing_per_token_ms/update_actor:0.040287025556094466 - timing_per_token_ms/gen:0.1717650301901359 - timing_per_token_ms/adv:0.0005131027484552061 - perf/total_num_tokens:251651 - perf/time_per_step:44.60779097676277 - perf/throughput:705.1767036925535 (TaskRunner pid=2049544) Training Progress: 47%|████▋ | 47/100 [34:10<36:52, 41.75s/it] (TaskRunner pid=2049544) step:48 - global_seqlen/min:18590 - global_seqlen/max:34823 - global_seqlen/minmax_diff:16233 - global_seqlen/balanced_min:28237 - global_seqlen/balanced_max:28801 - global_seqlen/mean:28407.875 - actor/entropy:0.14581364393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068536281585693 - training/rollout_probs_diff_mean:0.00291546992957592 - training/rollout_probs_diff_std:0.01052320096641779 - training/rollout_actor_probs_pearson_corr:0.9985108971595764 - rollout_corr/rollout_is_mean:0.9999670386314392 - rollout_corr/rollout_is_ratio_fraction_high:1.2597076420206577e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.779122926061973e-05 - rollout_corr/rollout_is_max:2.1663451194763184 - rollout_corr/rollout_is_min:0.35977837443351746 - rollout_corr/rollout_is_std:0.030446205288171768 - rollout_corr/rollout_is_eff_sample_size:0.9990738870447088 - rollout_corr/rollout_is_seq_mean:0.9999531507492065 - rollout_corr/rollout_is_seq_std:0.0016499547054991126 - rollout_corr/rollout_is_seq_max:1.0073652267456055 - rollout_corr/rollout_is_seq_min:0.9916918873786926 - rollout_corr/rollout_is_seq_max_deviation:0.008308112621307373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1378675876185298) - rollout_corr/training_log_ppl:np.float64(0.12675013709667837) - rollout_corr/kl:np.float64(0.000753273961016987) - rollout_corr/k3_kl:np.float64(0.0006820533353391056) - rollout_corr/rollout_ppl:np.float64(1.1369990683160722) - rollout_corr/rollout_log_ppl:np.float64(0.12599686199246207) - rollout_corr/log_ppl_diff:np.float64(0.0007532751042163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014732455310877413) - rollout_corr/log_ppl_diff_max:np.float64(0.009199053049087524) - rollout_corr/log_ppl_diff_min:np.float64(-0.006944321095943451) - rollout_corr/ppl_ratio:np.float64(1.0007553710602224) - rollout_corr/chi2_token:np.float64(0.0013299805577844381) - rollout_corr/chi2_seq:np.float64(0.6524860400240868) - actor/pg_loss:np.float64(-2.4408800527453423e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820084601917188) - actor/ppo_kl:np.float64(0.00018927762639009416) - actor/pg_clipfrac_lower:np.float64(1.1097301467088982e-05) - actor/grad_norm:np.float64(0.2395218200981617) - perf/mfu/actor:np.float64(0.08095804530180152) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.21407699584961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01809648796916008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01809648796916008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:620.18359375 - response_length/max:3666.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:620.18359375 - response_length_non_aborted/max:3666.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014730170369148254 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9394192546606064) - timing_s/agent_loop/generate_sequences/max:np.float64(21.90096348337829) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9833613064474775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.90096348337829) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3666 - timing_s/gen:28.045954892411828 - timing_s/reward:0.0676717646420002 - timing_s/old_log_prob:2.6066879238933325 - timing_s/adv:0.12023183144629002 - timing_s/update_actor:10.316766967996955 - timing_s/step:41.24971695430577 - timing_s/stop_profile:0.00013453327119350433 - timing_per_token_ms/update_actor:0.04539571759589971 - timing_per_token_ms/gen:0.17664851570170015 - timing_per_token_ms/adv:0.0005290427013912956 - perf/total_num_tokens:227263 - perf/time_per_step:41.24971695430577 - perf/throughput:688.680483104132 (TaskRunner pid=2049544) Training Progress: 48%|████▊ | 48/100 [34:51<36:03, 41.61s/it] (TaskRunner pid=2049544) step:49 - global_seqlen/min:18126 - global_seqlen/max:38899 - global_seqlen/minmax_diff:20773 - global_seqlen/balanced_min:26338 - global_seqlen/balanced_max:26710 - global_seqlen/mean:26451.75 - actor/entropy:0.18256059288978577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3656392991542816 - training/rollout_probs_diff_mean:0.0032566494774073362 - training/rollout_probs_diff_std:0.010963589884340763 - training/rollout_actor_probs_pearson_corr:0.9987143278121948 - rollout_corr/rollout_is_mean:1.0001035928726196 - rollout_corr/rollout_is_ratio_fraction_high:6.82920153849409e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.146281521068886e-05 - rollout_corr/rollout_is_max:2.4023168087005615 - rollout_corr/rollout_is_min:0.14749553799629211 - rollout_corr/rollout_is_std:0.03197216987609863 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0002081394195557 - rollout_corr/rollout_is_seq_std:0.001892627333290875 - rollout_corr/rollout_is_seq_max:1.008901834487915 - rollout_corr/rollout_is_seq_min:0.9917373061180115 - rollout_corr/rollout_is_seq_max_deviation:0.008901834487915039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1714685205370188) - rollout_corr/training_log_ppl:np.float64(0.15115834452444687) - rollout_corr/kl:np.float64(0.0004743365148804912) - rollout_corr/k3_kl:np.float64(0.0006857500739982925) - rollout_corr/rollout_ppl:np.float64(1.1708842911757529) - rollout_corr/rollout_log_ppl:np.float64(0.15068400763266254) - rollout_corr/log_ppl_diff:np.float64(0.0004743368917843327) - rollout_corr/log_ppl_abs_diff:np.float64(0.001488212525146082) - rollout_corr/log_ppl_diff_max:np.float64(0.010617084801197052) - rollout_corr/log_ppl_diff_min:np.float64(-0.015819266438484192) - rollout_corr/ppl_ratio:np.float64(1.0004769298247993) - rollout_corr/chi2_token:np.float64(0.0018537091091275215) - rollout_corr/chi2_seq:np.float64(0.6309683343861252) - actor/pg_loss:np.float64(3.3562071621418e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005490989437930693) - actor/ppo_kl:np.float64(8.648514501530258e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2604895681142807) - perf/mfu/actor:np.float64(0.07564944214215971) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27551984786987) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015421191230416298 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015421191230416298 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.9921875 - response_length/max:3613.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.9921875 - response_length_non_aborted/max:3613.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014311634004116058 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9619488045573235) - timing_s/agent_loop/generate_sequences/max:np.float64(21.043767675757408) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.410715061574592) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.043767675757408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:3586 - timing_s/gen:27.190765019506216 - timing_s/reward:0.06571215577423573 - timing_s/old_log_prob:2.4588573779910803 - timing_s/adv:0.11915759928524494 - timing_s/update_actor:10.26756145618856 - timing_s/step:40.190888445824385 - timing_s/stop_profile:0.00011413171887397766 - timing_per_token_ms/update_actor:0.048520237111857246 - timing_per_token_ms/gen:0.185691217779869 - timing_per_token_ms/adv:0.0005630893952443834 - perf/total_num_tokens:211614 - perf/time_per_step:40.190888445824385 - perf/throughput:658.1529053694804 (TaskRunner pid=2049544) Training Progress: 49%|████▉ | 49/100 [35:31<35:01, 41.20s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2049544) step:50 - global_seqlen/min:22621 - global_seqlen/max:38182 - global_seqlen/minmax_diff:15561 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:32242 - global_seqlen/mean:31432.0 - actor/entropy:0.20185792446136475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018432855606079 - training/rollout_probs_diff_mean:0.003106999211013317 - training/rollout_probs_diff_std:0.010115313343703747 - training/rollout_actor_probs_pearson_corr:0.9989494681358337 - rollout_corr/rollout_is_mean:1.0000050067901611 - rollout_corr/rollout_is_ratio_fraction_high:5.458515261125285e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.458515261125285e-06 - rollout_corr/rollout_is_max:2.0969016551971436 - rollout_corr/rollout_is_min:0.33815088868141174 - rollout_corr/rollout_is_std:0.030253741890192032 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:0.9999303817749023 - rollout_corr/rollout_is_seq_std:0.001567038125358522 - rollout_corr/rollout_is_seq_max:1.0050349235534668 - rollout_corr/rollout_is_seq_min:0.993672788143158 - rollout_corr/rollout_is_seq_max_deviation:0.006327211856842041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.191426008939743) - rollout_corr/training_log_ppl:np.float64(0.16859979891887633) - rollout_corr/kl:np.float64(0.0005692955341736194) - rollout_corr/k3_kl:np.float64(0.0005476311655883137) - rollout_corr/rollout_ppl:np.float64(1.1907334956340492) - rollout_corr/rollout_log_ppl:np.float64(0.16803050318412716) - rollout_corr/log_ppl_diff:np.float64(0.0005692957347491756) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012659797939704731) - rollout_corr/log_ppl_diff_max:np.float64(0.008102566003799438) - rollout_corr/log_ppl_diff_min:np.float64(-0.00437454879283905) - rollout_corr/ppl_ratio:np.float64(1.0005708439275622) - rollout_corr/chi2_token:np.float64(0.0010575931519269943) - rollout_corr/chi2_seq:np.float64(1.1911898551043123) - actor/pg_loss:np.float64(6.677815690636635e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036017004458699375) - actor/ppo_kl:np.float64(2.6112116451315615e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17873636819422245) - perf/mfu/actor:np.float64(0.0885810936633662) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.26837921142578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/score/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/score/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06833156224087977) - val-core/sciknoweval/acc/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09966534252010995) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8278625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06058065805524353) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6779125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.978125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05947208208072883) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9967624999999998) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02126663775002429) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9593875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0761225622474269) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9781625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.059091934676165636) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0017437187447881502) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9295375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08943279499033274) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9918500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03609564352248361) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.890025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.09147875280510498) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9984124999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014058383946768527) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8474) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07545396833360063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99985) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002693900935493915) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016236353665590286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016236353665590286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:715.625 - response_length/max:3780.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:715.625 - response_length_non_aborted/max:3780.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014733336865901947 - timing_s/agent_loop/generate_sequences/min:np.float64(0.98030811175704) - timing_s/agent_loop/generate_sequences/max:np.float64(23.301443008705974) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.870220143442566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.301443008705974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:3780 - timing_s/gen:29.260056165978312 - timing_s/reward:0.07223350368440151 - timing_s/old_log_prob:2.538018411025405 - timing_s/adv:0.11729281768202782 - timing_s/update_actor:10.420197248458862 - timing_s/step:42.49460377730429 - timing_s/testing:117.82088489830494 - timing_s/save_checkpoint:6.728960711508989 - timing_s/stop_profile:0.00011482089757919312 - timing_per_token_ms/update_actor:0.04143944566229822 - timing_per_token_ms/gen:0.15971646378809123 - timing_per_token_ms/adv:0.0004664546389110931 - perf/total_num_tokens:251456 - perf/time_per_step:42.49460377730429 - perf/throughput:739.6703864971049 (TaskRunner pid=2049544) Training Progress: 50%|█████ | 50/100 [38:18<1:05:48, 78.97s/it] (TaskRunner pid=2049544) step:51 - global_seqlen/min:18785 - global_seqlen/max:33808 - global_seqlen/minmax_diff:15023 - global_seqlen/balanced_min:26399 - global_seqlen/balanced_max:26471 - global_seqlen/mean:26450.125 - actor/entropy:0.24430422484874725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564788341522217 - training/rollout_probs_diff_mean:0.003582285949960351 - training/rollout_probs_diff_std:0.010883934795856476 - training/rollout_actor_probs_pearson_corr:0.999004602432251 - rollout_corr/rollout_is_mean:0.999866783618927 - rollout_corr/rollout_is_ratio_fraction_high:1.396638253936544e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.189914761809632e-05 - rollout_corr/rollout_is_max:2.2966716289520264 - rollout_corr/rollout_is_min:0.327895849943161 - rollout_corr/rollout_is_std:0.033104244619607925 - rollout_corr/rollout_is_eff_sample_size:0.9989051897151183 - rollout_corr/rollout_is_seq_mean:0.9998487830162048 - rollout_corr/rollout_is_seq_std:0.0018385471776127815 - rollout_corr/rollout_is_seq_max:1.0075657367706299 - rollout_corr/rollout_is_seq_min:0.9936858415603638 - rollout_corr/rollout_is_seq_max_deviation:0.007565736770629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.222205103840679) - rollout_corr/training_log_ppl:np.float64(0.18761249752424192) - rollout_corr/kl:np.float64(0.0007189881607398974) - rollout_corr/k3_kl:np.float64(0.0005682342473676272) - rollout_corr/rollout_ppl:np.float64(1.2213212260976434) - rollout_corr/rollout_log_ppl:np.float64(0.1868935083039105) - rollout_corr/log_ppl_diff:np.float64(0.000718989220331423) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408860151888803) - rollout_corr/log_ppl_diff_max:np.float64(0.007139421999454498) - rollout_corr/log_ppl_diff_min:np.float64(-0.005957402288913727) - rollout_corr/ppl_ratio:np.float64(1.0007210047915578) - rollout_corr/chi2_token:np.float64(0.0008357972837984562) - rollout_corr/chi2_seq:np.float64(1.0313024234492332) - actor/pg_loss:np.float64(4.522060044109821e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003210717095498694) - actor/ppo_kl:np.float64(-9.128155866733323e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14451957494020462) - perf/mfu/actor:np.float64(0.07523053933355461) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.30613327026367) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011186200194060802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011186200194060802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:559.37890625 - response_length/max:2702.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:559.37890625 - response_length_non_aborted/max:2702.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.59448516368866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9070874620229006) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53190640732646) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.214220573740022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53190640732646) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:2702 - timing_s/gen:22.698202308267355 - timing_s/reward:0.06384330801665783 - timing_s/old_log_prob:2.640516960993409 - timing_s/adv:0.15581872314214706 - timing_s/update_actor:10.303586572408676 - timing_s/step:35.9570118393749 - timing_s/stop_profile:0.00012137554585933685 - timing_per_token_ms/update_actor:0.048693468236958595 - timing_per_token_ms/gen:0.1585058924746849 - timing_per_token_ms/adv:0.0007363798996325493 - perf/total_num_tokens:211601 - perf/time_per_step:35.9570118393749 - perf/throughput:735.6040907447059 (TaskRunner pid=2049544) Training Progress: 51%|█████ | 51/100 [38:54<53:58, 66.09s/it] (TaskRunner pid=2049544) step:52 - global_seqlen/min:23429 - global_seqlen/max:36160 - global_seqlen/minmax_diff:12731 - global_seqlen/balanced_min:30668 - global_seqlen/balanced_max:30765 - global_seqlen/mean:30739.125 - actor/entropy:0.22692662477493286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3938141167163849 - training/rollout_probs_diff_mean:0.0033940812572836876 - training/rollout_probs_diff_std:0.010467931628227234 - training/rollout_actor_probs_pearson_corr:0.9990243315696716 - rollout_corr/rollout_is_mean:1.000043272972107 - rollout_corr/rollout_is_ratio_fraction_high:1.1359441487002186e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2718882974004373e-05 - rollout_corr/rollout_is_max:2.0554285049438477 - rollout_corr/rollout_is_min:0.29720351099967957 - rollout_corr/rollout_is_std:0.03246244043111801 - rollout_corr/rollout_is_eff_sample_size:0.9989472992270986 - rollout_corr/rollout_is_seq_mean:1.0000196695327759 - rollout_corr/rollout_is_seq_std:0.001915202010422945 - rollout_corr/rollout_is_seq_max:1.0059826374053955 - rollout_corr/rollout_is_seq_min:0.9903362989425659 - rollout_corr/rollout_is_seq_max_deviation:0.009663701057434082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.231195269152522) - rollout_corr/training_log_ppl:np.float64(0.19834982651809696) - rollout_corr/kl:np.float64(0.0008127379676388102) - rollout_corr/k3_kl:np.float64(0.0006191496557619303) - rollout_corr/rollout_ppl:np.float64(1.230196955613792) - rollout_corr/rollout_log_ppl:np.float64(0.19753708717325935) - rollout_corr/log_ppl_diff:np.float64(0.0008127393448376097) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014782494035898708) - rollout_corr/log_ppl_diff_max:np.float64(0.011166423559188843) - rollout_corr/log_ppl_diff_min:np.float64(-0.00570167601108551) - rollout_corr/ppl_ratio:np.float64(1.0008151046931744) - rollout_corr/chi2_token:np.float64(0.0008716660086065531) - rollout_corr/chi2_seq:np.float64(0.30706389155238867) - actor/pg_loss:np.float64(2.5613000616431236e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015267897435933264) - actor/ppo_kl:np.float64(0.00024156836637168055) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18445459008216858) - perf/mfu/actor:np.float64(0.08808419007285372) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.37739562988281) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007727969903498888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007727969903498888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:687.75390625 - response_length/max:2764.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:687.75390625 - response_length_non_aborted/max:2764.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011474080383777618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7448907978832722) - timing_s/agent_loop/generate_sequences/max:np.float64(18.379133939743042) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.580681977618951) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.379133939743042) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2764 - timing_s/gen:24.19850930571556 - timing_s/reward:0.07030453719198704 - timing_s/old_log_prob:2.519341915845871 - timing_s/adv:0.11268062144517899 - timing_s/update_actor:10.240152211859822 - timing_s/step:37.230506498366594 - timing_s/stop_profile:0.00011861883103847504 - timing_per_token_ms/update_actor:0.0416413618306467 - timing_per_token_ms/gen:0.13744077077054248 - timing_per_token_ms/adv:0.0004582133577532663 - perf/total_num_tokens:245913 - perf/time_per_step:37.230506498366594 - perf/throughput:825.643481410832 (TaskRunner pid=2049544) Training Progress: 52%|█████▏ | 52/100 [39:32<45:57, 57.45s/it] (TaskRunner pid=2049544) step:53 - global_seqlen/min:19572 - global_seqlen/max:34404 - global_seqlen/minmax_diff:14832 - global_seqlen/balanced_min:27920 - global_seqlen/balanced_max:29013 - global_seqlen/mean:28188.25 - actor/entropy:0.22438420355319977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30675768852233887 - training/rollout_probs_diff_mean:0.0035303884651511908 - training/rollout_probs_diff_std:0.010939665138721466 - training/rollout_actor_probs_pearson_corr:0.9988002181053162 - rollout_corr/rollout_is_mean:1.000083327293396 - rollout_corr/rollout_is_ratio_fraction_high:1.938861169037409e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.524009636952542e-05 - rollout_corr/rollout_is_max:2.4050400257110596 - rollout_corr/rollout_is_min:0.11861640214920044 - rollout_corr/rollout_is_std:0.03258340060710907 - rollout_corr/rollout_is_eff_sample_size:0.9989399238583818 - rollout_corr/rollout_is_seq_mean:1.0001201629638672 - rollout_corr/rollout_is_seq_std:0.001550388871692121 - rollout_corr/rollout_is_seq_max:1.0046480894088745 - rollout_corr/rollout_is_seq_min:0.994028627872467 - rollout_corr/rollout_is_seq_max_deviation:0.005971372127532959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2066223253495991) - rollout_corr/training_log_ppl:np.float64(0.17772597989824135) - rollout_corr/kl:np.float64(0.0005446721567352597) - rollout_corr/k3_kl:np.float64(0.0006030060052921726) - rollout_corr/rollout_ppl:np.float64(1.2059485423378646) - rollout_corr/rollout_log_ppl:np.float64(0.17718130597495474) - rollout_corr/log_ppl_diff:np.float64(0.0005446739232866094) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306041274801828) - rollout_corr/log_ppl_diff_max:np.float64(0.006498947739601135) - rollout_corr/log_ppl_diff_min:np.float64(-0.005862616002559662) - rollout_corr/ppl_ratio:np.float64(1.0005462081171572) - rollout_corr/chi2_token:np.float64(0.001310169231146574) - rollout_corr/chi2_seq:np.float64(0.6170898247510195) - actor/pg_loss:np.float64(5.588738713413477e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040137838948339777) - actor/ppo_kl:np.float64(0.00010251342614253645) - actor/pg_clipfrac_lower:np.float64(1.1289739632047713e-06) - actor/grad_norm:np.float64(0.2714943513274193) - perf/mfu/actor:np.float64(0.0789630435605575) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.31622695922852) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02330026589334011 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02330026589334011 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.4140625 - response_length/max:3739.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.4140625 - response_length_non_aborted/max:3739.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001577865332365036 - timing_s/agent_loop/generate_sequences/min:np.float64(1.263271963223815) - timing_s/agent_loop/generate_sequences/max:np.float64(22.0587509367615) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.729596399491129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.0587509367615) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3739 - timing_s/gen:27.900284377858043 - timing_s/reward:0.0937537681311369 - timing_s/old_log_prob:2.7064144033938646 - timing_s/adv:0.1434136312454939 - timing_s/update_actor:10.502349453046918 - timing_s/step:41.45142296142876 - timing_s/stop_profile:0.00011624768376350403 - timing_per_token_ms/update_actor:0.04657237258896401 - timing_per_token_ms/gen:0.18031593341858748 - timing_per_token_ms/adv:0.0006359637049368704 - perf/total_num_tokens:225506 - perf/time_per_step:41.45142296142876 - perf/throughput:680.030937085794 (TaskRunner pid=2049544) Training Progress: 53%|█████▎ | 53/100 [40:13<41:15, 52.67s/it] (TaskRunner pid=2049544) step:54 - global_seqlen/min:21471 - global_seqlen/max:31367 - global_seqlen/minmax_diff:9896 - global_seqlen/balanced_min:26297 - global_seqlen/balanced_max:26328 - global_seqlen/mean:26318.0 - actor/entropy:0.21818441152572632 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292486190795898 - training/rollout_probs_diff_mean:0.0032305221538990736 - training/rollout_probs_diff_std:0.010491388849914074 - training/rollout_actor_probs_pearson_corr:0.9989482760429382 - rollout_corr/rollout_is_mean:1.0000078678131104 - rollout_corr/rollout_is_ratio_fraction_high:2.0765270164702088e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3843512533640023e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.26139989495277405 - rollout_corr/rollout_is_std:0.03054003044962883 - rollout_corr/rollout_is_eff_sample_size:0.999068175625075 - rollout_corr/rollout_is_seq_mean:0.9999727010726929 - rollout_corr/rollout_is_seq_std:0.001957092434167862 - rollout_corr/rollout_is_seq_max:1.007343053817749 - rollout_corr/rollout_is_seq_min:0.9894382953643799 - rollout_corr/rollout_is_seq_max_deviation:0.010561704635620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1926028821617365) - rollout_corr/training_log_ppl:np.float64(0.16857526991225313) - rollout_corr/kl:np.float64(0.0005841684274781045) - rollout_corr/k3_kl:np.float64(0.0005794948536674838) - rollout_corr/rollout_ppl:np.float64(1.1918872729875147) - rollout_corr/rollout_log_ppl:np.float64(0.1679911007740884) - rollout_corr/log_ppl_diff:np.float64(0.0005841691381647252) - rollout_corr/log_ppl_abs_diff:np.float64(0.001538367178000044) - rollout_corr/log_ppl_diff_max:np.float64(0.011501207947731018) - rollout_corr/log_ppl_diff_min:np.float64(-0.005205966532230377) - rollout_corr/ppl_ratio:np.float64(1.0005866582505405) - rollout_corr/chi2_token:np.float64(0.0011341231875121593) - rollout_corr/chi2_seq:np.float64(1.4627508698031306) - actor/pg_loss:np.float64(0.00015633960720151663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042431396536812827) - actor/ppo_kl:np.float64(3.235917171728886e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16759618930518627) - perf/mfu/actor:np.float64(0.07638956882275981) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.35180282592773) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008146907202899456 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008146907202899456 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:564.34375 - response_length/max:2093.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:564.34375 - response_length_non_aborted/max:2093.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015513785183429718 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6626291759312153) - timing_s/agent_loop/generate_sequences/max:np.float64(14.199840543791652) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.107884573328192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.199840543791652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:2093 - timing_s/gen:19.986812436953187 - timing_s/reward:0.06590797752141953 - timing_s/old_log_prob:2.560090212151408 - timing_s/adv:0.13934014923870564 - timing_s/update_actor:10.21920620650053 - timing_s/step:33.05696338787675 - timing_s/stop_profile:0.00011862441897392273 - timing_per_token_ms/update_actor:0.04853715236007927 - timing_per_token_ms/gen:0.13834384819863493 - timing_per_token_ms/adv:0.0006618101168340378 - perf/total_num_tokens:210544 - perf/time_per_step:33.05696338787675 - perf/throughput:796.1408823670663 (TaskRunner pid=2049544) Training Progress: 54%|█████▍ | 54/100 [40:46<35:52, 46.80s/it] (TaskRunner pid=2049544) step:55 - global_seqlen/min:18191 - global_seqlen/max:33987 - global_seqlen/minmax_diff:15796 - global_seqlen/balanced_min:26610 - global_seqlen/balanced_max:26684 - global_seqlen/mean:26660.125 - actor/entropy:0.1736723929643631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.0030401991680264473 - training/rollout_probs_diff_std:0.01039278693497181 - training/rollout_actor_probs_pearson_corr:0.9987287521362305 - rollout_corr/rollout_is_mean:1.0000579357147217 - rollout_corr/rollout_is_ratio_fraction_high:1.375430656480603e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.375430656480603e-05 - rollout_corr/rollout_is_max:2.335618734359741 - rollout_corr/rollout_is_min:0.392137348651886 - rollout_corr/rollout_is_std:0.030324585735797882 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.000092625617981 - rollout_corr/rollout_is_seq_std:0.00183792260941118 - rollout_corr/rollout_is_seq_max:1.0104801654815674 - rollout_corr/rollout_is_seq_min:0.9918577671051025 - rollout_corr/rollout_is_seq_max_deviation:0.010480165481567383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1669259667396545) - rollout_corr/training_log_ppl:np.float64(0.14700542857826804) - rollout_corr/kl:np.float64(0.0007857797350844242) - rollout_corr/k3_kl:np.float64(0.0007165672559068526) - rollout_corr/rollout_ppl:np.float64(1.1659750957041979) - rollout_corr/rollout_log_ppl:np.float64(0.14621964823891176) - rollout_corr/log_ppl_diff:np.float64(0.000785780339356279) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016149269104062114) - rollout_corr/log_ppl_diff_max:np.float64(0.058644767850637436) - rollout_corr/log_ppl_diff_min:np.float64(-0.004740942269563675) - rollout_corr/ppl_ratio:np.float64(1.00079559488222) - rollout_corr/chi2_token:np.float64(0.0010453276336193085) - rollout_corr/chi2_seq:np.float64(0.6579223412554711) - actor/pg_loss:np.float64(-0.00018631992861628532) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005601525168685839) - actor/ppo_kl:np.float64(0.00037079197026113064) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25370683521032333) - perf/mfu/actor:np.float64(0.0767427855529022) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.34751892089844) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002467178856022656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002467178856022656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.00390625 - response_length/max:2618.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.00390625 - response_length_non_aborted/max:2618.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001354403793811798 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8230457752943039) - timing_s/agent_loop/generate_sequences/max:np.float64(16.547815980389714) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.079917931871023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.547815980389714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2618 - timing_s/gen:22.890313317999244 - timing_s/reward:0.06661977060139179 - timing_s/old_log_prob:2.436913600191474 - timing_s/adv:0.1175360195338726 - timing_s/update_actor:10.17019310221076 - timing_s/step:35.7683734241873 - timing_s/stop_profile:0.0001267455518245697 - timing_per_token_ms/update_actor:0.04768447776506468 - timing_per_token_ms/gen:0.15742019626019876 - timing_per_token_ms/adv:0.0005510852796726975 - perf/total_num_tokens:213281 - perf/time_per_step:35.7683734241873 - perf/throughput:745.3546932042451 (TaskRunner pid=2049544) Training Progress: 55%|█████▌ | 55/100 [41:22<32:37, 43.51s/it] (TaskRunner pid=2049544) step:56 - global_seqlen/min:22164 - global_seqlen/max:48288 - global_seqlen/minmax_diff:26124 - global_seqlen/balanced_min:28371 - global_seqlen/balanced_max:28431 - global_seqlen/mean:28409.375 - actor/entropy:0.21692201495170593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174453377723694 - training/rollout_probs_diff_mean:0.003212681272998452 - training/rollout_probs_diff_std:0.010248677805066109 - training/rollout_actor_probs_pearson_corr:0.9989055395126343 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:1.2312617400311865e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.309416181058623e-05 - rollout_corr/rollout_is_max:2.551107883453369 - rollout_corr/rollout_is_min:0.3708029091358185 - rollout_corr/rollout_is_std:0.031356628984212875 - rollout_corr/rollout_is_eff_sample_size:0.9990177275875637 - rollout_corr/rollout_is_seq_mean:1.0000814199447632 - rollout_corr/rollout_is_seq_std:0.002115494105964899 - rollout_corr/rollout_is_seq_max:1.0103763341903687 - rollout_corr/rollout_is_seq_min:0.9925733208656311 - rollout_corr/rollout_is_seq_max_deviation:0.010376334190368652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2074726885184646) - rollout_corr/training_log_ppl:np.float64(0.18105585960438475) - rollout_corr/kl:np.float64(0.0007089410478542391) - rollout_corr/k3_kl:np.float64(0.0007255429266592728) - rollout_corr/rollout_ppl:np.float64(1.2066095937043428) - rollout_corr/rollout_log_ppl:np.float64(0.18034691788489) - rollout_corr/log_ppl_diff:np.float64(0.00070894171949476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016144337132573128) - rollout_corr/log_ppl_diff_max:np.float64(0.014080479741096497) - rollout_corr/log_ppl_diff_min:np.float64(-0.00835387408733368) - rollout_corr/ppl_ratio:np.float64(1.0007117404602468) - rollout_corr/chi2_token:np.float64(0.0016396085266023874) - rollout_corr/chi2_seq:np.float64(0.7452940770890564) - actor/pg_loss:np.float64(0.00023605907335877419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041431330055274884) - actor/ppo_kl:np.float64(0.00019277200499256963) - actor/pg_clipfrac_lower:np.float64(1.5439723938470706e-05) - actor/grad_norm:np.float64(0.27160290628671646) - perf/mfu/actor:np.float64(0.08149403640203684) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.39385986328125) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020954536739736795 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020954536739736795 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:634.51171875 - response_length/max:2697.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:634.51171875 - response_length_non_aborted/max:2697.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.834898471832275e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9801836125552654) - timing_s/agent_loop/generate_sequences/max:np.float64(17.489308523014188) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.853927971802477) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.489308523014188) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2659 - timing_s/gen:23.2441786210984 - timing_s/reward:0.08269395306706429 - timing_s/old_log_prob:2.530390776693821 - timing_s/adv:0.136021314188838 - timing_s/update_actor:10.16875221952796 - timing_s/step:36.25585407577455 - timing_s/stop_profile:0.00013040564954280853 - timing_per_token_ms/update_actor:0.04474206234529957 - timing_per_token_ms/gen:0.14309833854217624 - timing_per_token_ms/adv:0.0005984877975529117 - perf/total_num_tokens:227275 - perf/time_per_step:36.25585407577455 - perf/throughput:783.5803547924854 (TaskRunner pid=2049544) Training Progress: 56%|█████▌ | 56/100 [41:59<30:19, 41.35s/it] (TaskRunner pid=2049544) step:57 - global_seqlen/min:25396 - global_seqlen/max:39989 - global_seqlen/minmax_diff:14593 - global_seqlen/balanced_min:32645 - global_seqlen/balanced_max:32684 - global_seqlen/mean:32673.25 - actor/entropy:0.20889751613140106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23668746650218964 - training/rollout_probs_diff_mean:0.0030729498248547316 - training/rollout_probs_diff_std:0.009970693849027157 - training/rollout_actor_probs_pearson_corr:0.9990144371986389 - rollout_corr/rollout_is_mean:0.9999489188194275 - rollout_corr/rollout_is_ratio_fraction_high:5.2733157644979656e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.691321035148576e-05 - rollout_corr/rollout_is_max:2.685802698135376 - rollout_corr/rollout_is_min:0.4615940749645233 - rollout_corr/rollout_is_std:0.03016890585422516 - rollout_corr/rollout_is_eff_sample_size:0.9990906647175409 - rollout_corr/rollout_is_seq_mean:0.9998156428337097 - rollout_corr/rollout_is_seq_std:0.0015230864519253373 - rollout_corr/rollout_is_seq_max:1.0045884847640991 - rollout_corr/rollout_is_seq_min:0.9938079118728638 - rollout_corr/rollout_is_seq_max_deviation:0.0061920881271362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2169137438759208) - rollout_corr/training_log_ppl:np.float64(0.18850286441738717) - rollout_corr/kl:np.float64(0.0008272747242870082) - rollout_corr/k3_kl:np.float64(0.0005567002334601057) - rollout_corr/rollout_ppl:np.float64(1.2158889258280396) - rollout_corr/rollout_log_ppl:np.float64(0.18767558728723088) - rollout_corr/log_ppl_diff:np.float64(0.0008272771301562898) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012787482482963242) - rollout_corr/log_ppl_diff_max:np.float64(0.00949886441230774) - rollout_corr/log_ppl_diff_min:np.float64(-0.003404974937438965) - rollout_corr/ppl_ratio:np.float64(1.000829064520076) - rollout_corr/chi2_token:np.float64(0.0005316368769854307) - rollout_corr/chi2_seq:np.float64(0.33462699479423463) - actor/pg_loss:np.float64(4.014407750219107e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038230796246807586) - actor/ppo_kl:np.float64(0.00015057841544852124) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21619294956326485) - perf/mfu/actor:np.float64(0.09399730782746046) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.41035079956055) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012718578800559044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012718578800559044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:740.7578125 - response_length/max:2402.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:740.7578125 - response_length_non_aborted/max:2402.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010364875197410583 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0282678361982107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.842207921668887) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.293985813957988) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.842207921668887) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:361 - timing_s/agent_loop/slowest/response_length:2402 - timing_s/gen:22.68095701187849 - timing_s/reward:0.07416719943284988 - timing_s/old_log_prob:2.519518541172147 - timing_s/adv:0.15157914347946644 - timing_s/update_actor:10.281581053510308 - timing_s/step:35.79420953243971 - timing_s/stop_profile:0.00011889636516571045 - timing_per_token_ms/update_actor:0.03933485746562673 - timing_per_token_ms/gen:0.11960385274728419 - timing_per_token_ms/adv:0.0005799053640189851 - perf/total_num_tokens:261386 - perf/time_per_step:35.79420953243971 - perf/throughput:912.8082566089012 (TaskRunner pid=2049544) Training Progress: 57%|█████▋ | 57/100 [42:34<28:26, 39.69s/it] (TaskRunner pid=2049544) step:58 - global_seqlen/min:19068 - global_seqlen/max:41428 - global_seqlen/minmax_diff:22360 - global_seqlen/balanced_min:32655 - global_seqlen/balanced_max:33260 - global_seqlen/mean:32761.875 - actor/entropy:0.25287628173828125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42262715101242065 - training/rollout_probs_diff_mean:0.0032485839910805225 - training/rollout_probs_diff_std:0.009784280322492123 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999250769615173 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.117093729088083e-05 - rollout_corr/rollout_is_max:1.704056739807129 - rollout_corr/rollout_is_min:0.30506160855293274 - rollout_corr/rollout_is_std:0.030359944328665733 - rollout_corr/rollout_is_eff_sample_size:0.9990791225700723 - rollout_corr/rollout_is_seq_mean:0.9998817443847656 - rollout_corr/rollout_is_seq_std:0.0017060940153896809 - rollout_corr/rollout_is_seq_max:1.0051213502883911 - rollout_corr/rollout_is_seq_min:0.9917200803756714 - rollout_corr/rollout_is_seq_max_deviation:0.008279919624328613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.248608773574233) - rollout_corr/training_log_ppl:np.float64(0.2106571579060983) - rollout_corr/kl:np.float64(0.0006302413064700474) - rollout_corr/k3_kl:np.float64(0.0005439822076880318) - rollout_corr/rollout_ppl:np.float64(1.2478208504617214) - rollout_corr/rollout_log_ppl:np.float64(0.21002691524336115) - rollout_corr/log_ppl_diff:np.float64(0.000630242662737146) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013482350041158497) - rollout_corr/log_ppl_diff_max:np.float64(0.007139943540096283) - rollout_corr/log_ppl_diff_min:np.float64(-0.009232334792613983) - rollout_corr/ppl_ratio:np.float64(1.0006320539396256) - rollout_corr/chi2_token:np.float64(0.0009030492510646582) - rollout_corr/chi2_seq:np.float64(2.417929224204272) - actor/pg_loss:np.float64(-9.389116894453764e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003193014416638107) - actor/ppo_kl:np.float64(-1.695539558355108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21667541190981865) - perf/mfu/actor:np.float64(0.09071205098155613) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.45818328857422) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013659363612532616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013659363612532616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:751.90234375 - response_length/max:4193.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.90234375 - response_length_non_aborted/max:4193.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.30633395910263e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7540517617017031) - timing_s/agent_loop/generate_sequences/max:np.float64(25.78281068429351) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.984791398346715) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.78281068429351) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:4193 - timing_s/gen:31.48099230416119 - timing_s/reward:0.07489402964711189 - timing_s/old_log_prob:2.5185533091425896 - timing_s/adv:0.12432781234383583 - timing_s/update_actor:10.674950456246734 - timing_s/step:44.96027087979019 - timing_s/stop_profile:0.00012622401118278503 - timing_per_token_ms/update_actor:0.04072931744690564 - timing_per_token_ms/gen:0.16354866720433686 - timing_per_token_ms/adv:0.00047436163354446225 - perf/total_num_tokens:262095 - perf/time_per_step:44.96027087979019 - perf/throughput:728.6850003104983 (TaskRunner pid=2049544) Training Progress: 58%|█████▊ | 58/100 [43:19<28:54, 41.29s/it] (TaskRunner pid=2049544) step:59 - global_seqlen/min:23178 - global_seqlen/max:36252 - global_seqlen/minmax_diff:13074 - global_seqlen/balanced_min:28566 - global_seqlen/balanced_max:28612 - global_seqlen/mean:28596.625 - actor/entropy:0.24240879714488983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797503709793091 - training/rollout_probs_diff_mean:0.0033322961535304785 - training/rollout_probs_diff_std:0.010173420421779156 - training/rollout_actor_probs_pearson_corr:0.9990172982215881 - rollout_corr/rollout_is_mean:1.0001167058944702 - rollout_corr/rollout_is_ratio_fraction_high:6.2438730310532264e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.873162000265438e-05 - rollout_corr/rollout_is_max:2.413045644760132 - rollout_corr/rollout_is_min:0.38240286707878113 - rollout_corr/rollout_is_std:0.03103756159543991 - rollout_corr/rollout_is_eff_sample_size:0.9990379537831949 - rollout_corr/rollout_is_seq_mean:1.0002449750900269 - rollout_corr/rollout_is_seq_std:0.0013697257963940501 - rollout_corr/rollout_is_seq_max:1.003889560699463 - rollout_corr/rollout_is_seq_min:0.9955812692642212 - rollout_corr/rollout_is_seq_max_deviation:0.004418730735778809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2532247374765575) - rollout_corr/training_log_ppl:np.float64(0.21026221416104818) - rollout_corr/kl:np.float64(0.00036410490782223803) - rollout_corr/k3_kl:np.float64(0.0005213920420086282) - rollout_corr/rollout_ppl:np.float64(1.2527254964224994) - rollout_corr/rollout_log_ppl:np.float64(0.20989810848550405) - rollout_corr/log_ppl_diff:np.float64(0.0003641056755441241) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011843454485642724) - rollout_corr/log_ppl_diff_max:np.float64(0.0051978230476379395) - rollout_corr/log_ppl_diff_min:np.float64(-0.004664614796638489) - rollout_corr/ppl_ratio:np.float64(1.000365313142538) - rollout_corr/chi2_token:np.float64(0.0013532442972064018) - rollout_corr/chi2_seq:np.float64(1.167969731381163) - actor/pg_loss:np.float64(5.0825648941099644e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027953678352332645) - actor/ppo_kl:np.float64(0.0001204940012513589) - actor/pg_clipfrac_lower:np.float64(2.0624340777430916e-06) - actor/grad_norm:np.float64(0.2315312847495079) - perf/mfu/actor:np.float64(0.08222550993023878) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56365966796875) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007726792828179896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007726792828179896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:625.61328125 - response_length/max:2175.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:625.61328125 - response_length_non_aborted/max:2175.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.861107587814331e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0195303186774254) - timing_s/agent_loop/generate_sequences/max:np.float64(15.18119196780026) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.312842677085428) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.18119196780026) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:2156 - timing_s/gen:21.4505631364882 - timing_s/reward:0.06381673738360405 - timing_s/old_log_prob:2.4540651980787516 - timing_s/adv:0.12210911512374878 - timing_s/update_actor:10.219576759263873 - timing_s/step:34.34806403145194 - timing_s/stop_profile:0.00012100301682949066 - timing_per_token_ms/update_actor:0.04467125385978185 - timing_per_token_ms/gen:0.13393459628045104 - timing_per_token_ms/adv:0.0005337566720012797 - perf/total_num_tokens:228773 - perf/time_per_step:34.34806403145194 - perf/throughput:832.5542008368959 (TaskRunner pid=2049544) Training Progress: 59%|█████▉ | 59/100 [43:54<26:48, 39.22s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2049544) step:60 - global_seqlen/min:19702 - global_seqlen/max:41022 - global_seqlen/minmax_diff:21320 - global_seqlen/balanced_min:29482 - global_seqlen/balanced_max:29782 - global_seqlen/mean:29534.0 - actor/entropy:0.2471179962158203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2885955274105072 - training/rollout_probs_diff_mean:0.0032362197525799274 - training/rollout_probs_diff_std:0.009918652474880219 - training/rollout_actor_probs_pearson_corr:0.9991265535354614 - rollout_corr/rollout_is_mean:1.0000226497650146 - rollout_corr/rollout_is_ratio_fraction_high:1.1968021681241225e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1968021681241225e-05 - rollout_corr/rollout_is_max:2.9771621227264404 - rollout_corr/rollout_is_min:0.3293769657611847 - rollout_corr/rollout_is_std:0.03091634064912796 - rollout_corr/rollout_is_eff_sample_size:0.999045211617763 - rollout_corr/rollout_is_seq_mean:0.9999569654464722 - rollout_corr/rollout_is_seq_std:0.0014956871746107936 - rollout_corr/rollout_is_seq_max:1.0054192543029785 - rollout_corr/rollout_is_seq_min:0.9950917959213257 - rollout_corr/rollout_is_seq_max_deviation:0.005419254302978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2401416734792292) - rollout_corr/training_log_ppl:np.float64(0.19959818758798065) - rollout_corr/kl:np.float64(0.0005349192640053957) - rollout_corr/k3_kl:np.float64(0.0004940067541099324) - rollout_corr/rollout_ppl:np.float64(1.2394655891694129) - rollout_corr/rollout_log_ppl:np.float64(0.19906326734781032) - rollout_corr/log_ppl_diff:np.float64(0.0005349202401703224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775598395615816) - rollout_corr/log_ppl_diff_max:np.float64(0.005937367677688599) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511777311563492) - rollout_corr/ppl_ratio:np.float64(1.000536180799827) - rollout_corr/chi2_token:np.float64(0.0009251013398170471) - rollout_corr/chi2_seq:np.float64(1.390581222018227) - actor/pg_loss:np.float64(4.893657751381397e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018130719581677113) - actor/ppo_kl:np.float64(-2.2069814953340483e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15835927985608578) - perf/mfu/actor:np.float64(0.08403507336330182) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.51084518432617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6375) - val-aux/sciknoweval/reward/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/score/mean@16:np.float64(0.6375) - val-aux/sciknoweval/score/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.076862250314467) - val-core/sciknoweval/acc/mean@16:np.float64(0.6375) - val-aux/sciknoweval/acc/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10267672443906181) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8312625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.052249733074818304) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6821875000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0574668098874074) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9954875000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.022722407353494737) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9575250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07239005753165481) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9770249999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05686531821374918) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995499999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004893080696751963) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9285500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08215960170640288) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9901500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03655386464803364) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8929375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08171927324120949) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971125000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.017114100398696087) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8558125000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06612062868202828) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00411263922012108) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00966417696326971 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.00966417696326971 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:652.78125 - response_length/max:3139.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:652.78125 - response_length_non_aborted/max:3139.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.66247171163559e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2835856825113297) - timing_s/agent_loop/generate_sequences/max:np.float64(20.483323162421584) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.35434487216844) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.483323162421584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:3139 - timing_s/gen:26.403195202350616 - timing_s/reward:0.06402513571083546 - timing_s/old_log_prob:2.5449625328183174 - timing_s/adv:0.11957781203091145 - timing_s/update_actor:10.040485488250852 - timing_s/step:39.238921258598566 - timing_s/testing:114.50520241633058 - timing_s/save_checkpoint:6.870979869738221 - timing_s/stop_profile:0.00015047751367092133 - timing_per_token_ms/update_actor:0.04249545222561646 - timing_per_token_ms/gen:0.15799700322149587 - timing_per_token_ms/adv:0.0005061023398071351 - perf/total_num_tokens:236272 - perf/time_per_step:39.238921258598566 - perf/throughput:752.6710483542691 (TaskRunner pid=2049544) Training Progress: 60%|██████ | 60/100 [46:34<50:26, 75.65s/it] (TaskRunner pid=2049544) step:61 - global_seqlen/min:22238 - global_seqlen/max:39486 - global_seqlen/minmax_diff:17248 - global_seqlen/balanced_min:29924 - global_seqlen/balanced_max:30089 - global_seqlen/mean:29970.875 - actor/entropy:0.23904968798160553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5477294921875 - training/rollout_probs_diff_mean:0.003164143767207861 - training/rollout_probs_diff_std:0.00985527690500021 - training/rollout_actor_probs_pearson_corr:0.9991676211357117 - rollout_corr/rollout_is_mean:1.0000258684158325 - rollout_corr/rollout_is_ratio_fraction_high:1.1471639481897e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.720745967759285e-05 - rollout_corr/rollout_is_max:4.02189826965332 - rollout_corr/rollout_is_min:0.26447275280952454 - rollout_corr/rollout_is_std:0.030506834387779236 - rollout_corr/rollout_is_eff_sample_size:0.9990700794243639 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.0016575142508372664 - rollout_corr/rollout_is_seq_max:1.0063189268112183 - rollout_corr/rollout_is_seq_min:0.9920255541801453 - rollout_corr/rollout_is_seq_max_deviation:0.007974445819854736 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25040641753003) - rollout_corr/training_log_ppl:np.float64(0.20799311240261886) - rollout_corr/kl:np.float64(0.0005623800304297433) - rollout_corr/k3_kl:np.float64(0.0005192289710862497) - rollout_corr/rollout_ppl:np.float64(1.2496482259593904) - rollout_corr/rollout_log_ppl:np.float64(0.20743073029007064) - rollout_corr/log_ppl_diff:np.float64(0.0005623821125482209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012774796996382065) - rollout_corr/log_ppl_diff_max:np.float64(0.009329631924629211) - rollout_corr/log_ppl_diff_min:np.float64(-0.007050305604934692) - rollout_corr/ppl_ratio:np.float64(1.0005640257149935) - rollout_corr/chi2_token:np.float64(0.0009777885861694813) - rollout_corr/chi2_seq:np.float64(0.3470004736445844) - actor/pg_loss:np.float64(-0.0002099055564031005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002631939267985217) - actor/ppo_kl:np.float64(6.142829495781044e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23834596201777458) - perf/mfu/actor:np.float64(0.08600884737044563) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.00771045684814) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007454414386302233 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.007454414386302233 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:681.02734375 - response_length/max:3128.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:681.02734375 - response_length_non_aborted/max:3128.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015644170343875885 - timing_s/agent_loop/generate_sequences/min:np.float64(0.93056246265769) - timing_s/agent_loop/generate_sequences/max:np.float64(20.83841021731496) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.5293213194963755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.83841021731496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3128 - timing_s/gen:26.53979305177927 - timing_s/reward:0.08145093731582165 - timing_s/old_log_prob:2.588765997439623 - timing_s/adv:0.11384812742471695 - timing_s/update_actor:10.279441229999065 - timing_s/step:39.70467999763787 - timing_s/stop_profile:0.0001134183257818222 - timing_per_token_ms/update_actor:0.04287262730066717 - timing_per_token_ms/gen:0.1522274656956647 - timing_per_token_ms/adv:0.0004748281766244602 - perf/total_num_tokens:239767 - perf/time_per_step:39.70467999763787 - perf/throughput:754.8448949036497 (TaskRunner pid=2049544) Training Progress: 61%|██████ | 61/100 [47:14<42:10, 64.88s/it] (TaskRunner pid=2049544) step:62 - global_seqlen/min:24330 - global_seqlen/max:43162 - global_seqlen/minmax_diff:18832 - global_seqlen/balanced_min:34742 - global_seqlen/balanced_max:34794 - global_seqlen/mean:34771.375 - actor/entropy:0.2732798457145691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4058573246002197 - training/rollout_probs_diff_mean:0.00327512389048934 - training/rollout_probs_diff_std:0.009542555548250675 - training/rollout_actor_probs_pearson_corr:0.9991864562034607 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.467516540287761e-05 - rollout_corr/rollout_is_max:1.9362671375274658 - rollout_corr/rollout_is_min:0.37468504905700684 - rollout_corr/rollout_is_std:0.030330481007695198 - rollout_corr/rollout_is_eff_sample_size:0.9990807884305588 - rollout_corr/rollout_is_seq_mean:0.9999213814735413 - rollout_corr/rollout_is_seq_std:0.0014323026407510042 - rollout_corr/rollout_is_seq_max:1.0065138339996338 - rollout_corr/rollout_is_seq_min:0.9906108975410461 - rollout_corr/rollout_is_seq_max_deviation:0.009389102458953857 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2884436114691198) - rollout_corr/training_log_ppl:np.float64(0.23855835630820366) - rollout_corr/kl:np.float64(0.0005924100263499099) - rollout_corr/k3_kl:np.float64(0.0005057925414178044) - rollout_corr/rollout_ppl:np.float64(1.2876463397406042) - rollout_corr/rollout_log_ppl:np.float64(0.23796594546729466) - rollout_corr/log_ppl_diff:np.float64(0.0005924108409089968) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011562938307179138) - rollout_corr/log_ppl_diff_max:np.float64(0.009115807712078094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0043695345520973206) - rollout_corr/ppl_ratio:np.float64(1.0005937188398093) - rollout_corr/chi2_token:np.float64(0.0008554272353649139) - rollout_corr/chi2_seq:np.float64(0.8644126462750137) - actor/pg_loss:np.float64(-5.7160970754921436e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029845362780633877) - actor/ppo_kl:np.float64(3.140159503445261e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22896912693977356) - perf/mfu/actor:np.float64(0.09839975519482527) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0562973022461) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022515371441841125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022515371441841125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:798.54296875 - response_length/max:3135.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:798.54296875 - response_length_non_aborted/max:3135.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014179199934005737 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8601270448416471) - timing_s/agent_loop/generate_sequences/max:np.float64(20.932036239653826) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.75760180906218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.932036239653826) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:3135 - timing_s/gen:27.040857832878828 - timing_s/reward:0.07787876017391682 - timing_s/old_log_prob:2.5947242341935635 - timing_s/adv:0.12395207956433296 - timing_s/update_actor:10.449679136276245 - timing_s/step:40.37802067771554 - timing_s/stop_profile:0.0001217275857925415 - timing_per_token_ms/update_actor:0.03756566693248486 - timing_per_token_ms/gen:0.1322763521104298 - timing_per_token_ms/adv:0.0004455966997434418 - perf/total_num_tokens:278171 - perf/time_per_step:40.37802067771554 - perf/throughput:861.1460991992154 (TaskRunner pid=2049544) Training Progress: 62%|██████▏ | 62/100 [47:55<36:26, 57.55s/it] (TaskRunner pid=2049544) step:63 - global_seqlen/min:21359 - global_seqlen/max:35870 - global_seqlen/minmax_diff:14511 - global_seqlen/balanced_min:26722 - global_seqlen/balanced_max:27254 - global_seqlen/mean:26812.0 - actor/entropy:0.27676644921302795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9831006526947021 - training/rollout_probs_diff_mean:0.0035199327394366264 - training/rollout_probs_diff_std:0.010530827566981316 - training/rollout_actor_probs_pearson_corr:0.9990900158882141 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:1.3679142284672707e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.419785571168177e-05 - rollout_corr/rollout_is_max:2.8213930130004883 - rollout_corr/rollout_is_min:0.011052249930799007 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990135634724805 - rollout_corr/rollout_is_seq_mean:1.0000073909759521 - rollout_corr/rollout_is_seq_std:0.001668434706516564 - rollout_corr/rollout_is_seq_max:1.0121815204620361 - rollout_corr/rollout_is_seq_min:0.9929383993148804 - rollout_corr/rollout_is_seq_max_deviation:0.012181520462036133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705106027424335) - rollout_corr/training_log_ppl:np.float64(0.22211689058167394) - rollout_corr/kl:np.float64(0.0004739185238458177) - rollout_corr/k3_kl:np.float64(0.0005098591802266128) - rollout_corr/rollout_ppl:np.float64(1.2698905346915126) - rollout_corr/rollout_log_ppl:np.float64(0.2216429725958733) - rollout_corr/log_ppl_diff:np.float64(0.0004739179858006537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012847277976106852) - rollout_corr/log_ppl_diff_max:np.float64(0.0076948776841163635) - rollout_corr/log_ppl_diff_min:np.float64(-0.010769776999950409) - rollout_corr/ppl_ratio:np.float64(1.0004755130503327) - rollout_corr/chi2_token:np.float64(0.001110320445150137) - rollout_corr/chi2_seq:np.float64(0.7298937705345452) - actor/pg_loss:np.float64(-6.4569758251309395e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017891856134610862) - actor/ppo_kl:np.float64(-4.981012998683809e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19124767929315567) - perf/mfu/actor:np.float64(0.07698739538780973) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.1104507446289) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018338599475100636 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0018338599475100636 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:571.125 - response_length/max:3010.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.125 - response_length_non_aborted/max:3010.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001445487141609192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8224132973700762) - timing_s/agent_loop/generate_sequences/max:np.float64(18.348940243944526) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.530061706544075) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.348940243944526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:3010 - timing_s/gen:25.11938327550888 - timing_s/reward:0.06868074834346771 - timing_s/old_log_prob:2.4906132221221924 - timing_s/adv:0.12919375859200954 - timing_s/update_actor:10.244162807241082 - timing_s/step:38.14616190828383 - timing_s/stop_profile:0.00011005811393260956 - timing_per_token_ms/update_actor:0.047759225380618206 - timing_per_token_ms/gen:0.17180580594433192 - timing_per_token_ms/adv:0.0006023131368044604 - perf/total_num_tokens:214496 - perf/time_per_step:38.14616190828383 - perf/throughput:702.8754312023591 (TaskRunner pid=2049544) Training Progress: 63%|██████▎ | 63/100 [48:33<31:54, 51.74s/it] (TaskRunner pid=2049544) step:64 - global_seqlen/min:26498 - global_seqlen/max:40322 - global_seqlen/minmax_diff:13824 - global_seqlen/balanced_min:31563 - global_seqlen/balanced_max:31618 - global_seqlen/mean:31602.875 - actor/entropy:0.28700023889541626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535178303718567 - training/rollout_probs_diff_mean:0.0033518956042826176 - training/rollout_probs_diff_std:0.009729590266942978 - training/rollout_actor_probs_pearson_corr:0.9992855787277222 - rollout_corr/rollout_is_mean:1.0000327825546265 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0833825399458874e-05 - rollout_corr/rollout_is_max:1.9838535785675049 - rollout_corr/rollout_is_min:0.4158313572406769 - rollout_corr/rollout_is_std:0.03056929260492325 - rollout_corr/rollout_is_eff_sample_size:0.9990665098066491 - rollout_corr/rollout_is_seq_mean:0.9999979734420776 - rollout_corr/rollout_is_seq_std:0.0013619516976177692 - rollout_corr/rollout_is_seq_max:1.0039030313491821 - rollout_corr/rollout_is_seq_min:0.9954256415367126 - rollout_corr/rollout_is_seq_max_deviation:0.0045743584632873535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2764493175782263) - rollout_corr/training_log_ppl:np.float64(0.22693806317693088) - rollout_corr/kl:np.float64(0.00046152420509582015) - rollout_corr/k3_kl:np.float64(0.0005074531937054871) - rollout_corr/rollout_ppl:np.float64(1.275848121382296) - rollout_corr/rollout_log_ppl:np.float64(0.22647653929016087) - rollout_corr/log_ppl_diff:np.float64(0.00046152388677001) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011273344280198216) - rollout_corr/log_ppl_diff_max:np.float64(0.00530514121055603) - rollout_corr/log_ppl_diff_min:np.float64(-0.005483843386173248) - rollout_corr/ppl_ratio:np.float64(1.0004626805894077) - rollout_corr/chi2_token:np.float64(0.0011098254472017288) - rollout_corr/chi2_seq:np.float64(1.8452809148002416) - actor/pg_loss:np.float64(-0.00011544267181307077) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002127377759961746) - actor/ppo_kl:np.float64(-4.476623224114462e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18739718943834305) - perf/mfu/actor:np.float64(0.08994342294998352) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.98906707763672) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013745415955781937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013745415955781937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:721.12109375 - response_length/max:2725.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:721.12109375 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.722448885440826e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.080433465540409) - timing_s/agent_loop/generate_sequences/max:np.float64(17.78822729177773) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.979315571334155) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.78822729177773) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:2560 - timing_s/gen:23.619467133656144 - timing_s/reward:0.07297545671463013 - timing_s/old_log_prob:2.6043873876333237 - timing_s/adv:0.14066699519753456 - timing_s/update_actor:10.391787450760603 - timing_s/step:36.91719730384648 - timing_s/stop_profile:0.0001245010644197464 - timing_per_token_ms/update_actor:0.041103014562601514 - timing_per_token_ms/gen:0.127944591124151 - timing_per_token_ms/adv:0.0005563852782283833 - perf/total_num_tokens:252823 - perf/time_per_step:36.91719730384648 - perf/throughput:856.047514655378 (TaskRunner pid=2049544) Training Progress: 64%|██████▍ | 64/100 [49:10<28:23, 47.32s/it] (TaskRunner pid=2049544) step:65 - global_seqlen/min:13817 - global_seqlen/max:44402 - global_seqlen/minmax_diff:30585 - global_seqlen/balanced_min:27087 - global_seqlen/balanced_max:27398 - global_seqlen/mean:27214.25 - actor/entropy:0.2703191936016083 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730000019073486 - training/rollout_probs_diff_mean:0.003337751142680645 - training/rollout_probs_diff_std:0.009712927974760532 - training/rollout_actor_probs_pearson_corr:0.999212384223938 - rollout_corr/rollout_is_mean:0.9999141097068787 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3306897017173469e-05 - rollout_corr/rollout_is_max:1.8171900510787964 - rollout_corr/rollout_is_min:0.37625378370285034 - rollout_corr/rollout_is_std:0.030340304598212242 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.001954729435965419 - rollout_corr/rollout_is_seq_max:1.0170458555221558 - rollout_corr/rollout_is_seq_min:0.9947062730789185 - rollout_corr/rollout_is_seq_max_deviation:0.01704585552215576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2311511896550655) - rollout_corr/training_log_ppl:np.float64(0.1959650574644911) - rollout_corr/kl:np.float64(0.0005068062779431948) - rollout_corr/k3_kl:np.float64(0.0005566824616494159) - rollout_corr/rollout_ppl:np.float64(1.2305298056453466) - rollout_corr/rollout_log_ppl:np.float64(0.1954582512989873) - rollout_corr/log_ppl_diff:np.float64(0.0005068061655038036) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013243403445812874) - rollout_corr/log_ppl_diff_max:np.float64(0.007357165217399597) - rollout_corr/log_ppl_diff_min:np.float64(-0.020653128623962402) - rollout_corr/ppl_ratio:np.float64(1.0005090332124382) - rollout_corr/chi2_token:np.float64(0.0013379347510635853) - rollout_corr/chi2_seq:np.float64(0.5852078427560627) - actor/pg_loss:np.float64(5.363975651562214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001393496023638363) - actor/ppo_kl:np.float64(4.77582690940892e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17264313623309135) - perf/mfu/actor:np.float64(0.07762399981838528) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82825469970703) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034032389521598816 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034032389521598816 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:587.1015625 - response_length/max:3738.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.1015625 - response_length_non_aborted/max:3738.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.999820470809937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0192465409636497) - timing_s/agent_loop/generate_sequences/max:np.float64(20.993049459531903) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.240891806744912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.993049459531903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:3738 - timing_s/gen:26.803196189925075 - timing_s/reward:0.06635245122015476 - timing_s/old_log_prob:2.5201703682541847 - timing_s/adv:0.12346276640892029 - timing_s/update_actor:10.165898667648435 - timing_s/step:39.771548522636294 - timing_s/stop_profile:0.00011387281119823456 - timing_per_token_ms/update_actor:0.04669382156245549 - timing_per_token_ms/gen:0.17833368501194344 - timing_per_token_ms/adv:0.0005670869416248853 - perf/total_num_tokens:217714 - perf/time_per_step:39.771548522636294 - perf/throughput:684.2642796397729 (TaskRunner pid=2049544) Training Progress: 65%|██████▌ | 65/100 [49:50<26:17, 45.07s/it] (TaskRunner pid=2049544) step:66 - global_seqlen/min:26398 - global_seqlen/max:47072 - global_seqlen/minmax_diff:20674 - global_seqlen/balanced_min:34031 - global_seqlen/balanced_max:34100 - global_seqlen/mean:34083.75 - actor/entropy:0.32043859362602234 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4526241421699524 - training/rollout_probs_diff_mean:0.003443863708525896 - training/rollout_probs_diff_std:0.009328007698059082 - training/rollout_actor_probs_pearson_corr:0.999345064163208 - rollout_corr/rollout_is_mean:0.999977171421051 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9896735466318205e-05 - rollout_corr/rollout_is_max:1.9577182531356812 - rollout_corr/rollout_is_min:0.13161711394786835 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:1.0000357627868652 - rollout_corr/rollout_is_seq_std:0.0014041649410501122 - rollout_corr/rollout_is_seq_max:1.0082844495773315 - rollout_corr/rollout_is_seq_min:0.9953852295875549 - rollout_corr/rollout_is_seq_max_deviation:0.008284449577331543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3104135780595243) - rollout_corr/training_log_ppl:np.float64(0.25584780907956883) - rollout_corr/kl:np.float64(0.0005620165548441136) - rollout_corr/k3_kl:np.float64(0.0004995921321437891) - rollout_corr/rollout_ppl:np.float64(1.309668519999832) - rollout_corr/rollout_log_ppl:np.float64(0.2552857908885926) - rollout_corr/log_ppl_diff:np.float64(0.0005620181909762323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011583589657675475) - rollout_corr/log_ppl_diff_max:np.float64(0.005684643983840942) - rollout_corr/log_ppl_diff_min:np.float64(-0.006929904222488403) - rollout_corr/ppl_ratio:np.float64(1.000563226873055) - rollout_corr/chi2_token:np.float64(0.0008636738639324903) - rollout_corr/chi2_seq:np.float64(0.8563104090280831) - actor/pg_loss:np.float64(0.0001647728495299816) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017283655381561402) - actor/ppo_kl:np.float64(0.00013175434309165013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1924263872206211) - perf/mfu/actor:np.float64(0.09599912196819169) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.91796875) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000762293697334826 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000762293697334826 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.3046875 - response_length/max:3216.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.3046875 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.582083344459534e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2348940391093493) - timing_s/agent_loop/generate_sequences/max:np.float64(21.131182158365846) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.555415921531676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.131182158365846) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3216 - timing_s/gen:26.812409596517682 - timing_s/reward:0.11229918338358402 - timing_s/old_log_prob:2.537730697542429 - timing_s/adv:0.12344273924827576 - timing_s/update_actor:10.477383820340037 - timing_s/step:40.15382357686758 - timing_s/stop_profile:0.00011853314936161041 - timing_per_token_ms/update_actor:0.03842514328800395 - timing_per_token_ms/gen:0.1333698584174021 - timing_per_token_ms/adv:0.00045271844811778253 - perf/total_num_tokens:272670 - perf/time_per_step:40.15382357686758 - perf/throughput:848.8295002530091 (TaskRunner pid=2049544) Training Progress: 66%|██████▌ | 66/100 [50:30<24:42, 43.61s/it] (TaskRunner pid=2049544) step:67 - global_seqlen/min:14820 - global_seqlen/max:39555 - global_seqlen/minmax_diff:24735 - global_seqlen/balanced_min:25556 - global_seqlen/balanced_max:26212 - global_seqlen/mean:25672.375 - actor/entropy:0.27359628677368164 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46210977435112 - training/rollout_probs_diff_mean:0.0033997194841504097 - training/rollout_probs_diff_std:0.009928795509040356 - training/rollout_actor_probs_pearson_corr:0.9991419315338135 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1833584469277412e-05 - rollout_corr/rollout_is_max:1.9785306453704834 - rollout_corr/rollout_is_min:0.3678695261478424 - rollout_corr/rollout_is_std:0.031190816313028336 - rollout_corr/rollout_is_eff_sample_size:0.9990281975159966 - rollout_corr/rollout_is_seq_mean:1.0001834630966187 - rollout_corr/rollout_is_seq_std:0.002221266273409128 - rollout_corr/rollout_is_seq_max:1.0159006118774414 - rollout_corr/rollout_is_seq_min:0.9894429445266724 - rollout_corr/rollout_is_seq_max_deviation:0.015900611877441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3023830708116293) - rollout_corr/training_log_ppl:np.float64(0.2422099069954129) - rollout_corr/kl:np.float64(0.00026683149832429365) - rollout_corr/k3_kl:np.float64(0.000541509634160775) - rollout_corr/rollout_ppl:np.float64(1.301996021065861) - rollout_corr/rollout_log_ppl:np.float64(0.241943074419396) - rollout_corr/log_ppl_diff:np.float64(0.0002668325760168955) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014319048641482368) - rollout_corr/log_ppl_diff_max:np.float64(0.009274959564208984) - rollout_corr/log_ppl_diff_min:np.float64(-0.0138968825340271) - rollout_corr/ppl_ratio:np.float64(1.0002689892426133) - rollout_corr/chi2_token:np.float64(0.0016502379439771175) - rollout_corr/chi2_seq:np.float64(1.019609775627032) - actor/pg_loss:np.float64(-5.0529371947050095e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016554813828406623) - actor/ppo_kl:np.float64(-7.827897409384832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17443475499749184) - perf/mfu/actor:np.float64(0.07407053273139065) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82710266113281) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01963839866220951 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01963839866220951 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:536.73046875 - response_length/max:3340.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:536.73046875 - response_length_non_aborted/max:3340.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023074820637702942 - timing_s/agent_loop/generate_sequences/min:np.float64(1.033557141199708) - timing_s/agent_loop/generate_sequences/max:np.float64(19.718276670202613) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.309584286827885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.718276670202613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3340 - timing_s/gen:25.747549824416637 - timing_s/reward:0.05820099823176861 - timing_s/old_log_prob:2.587189307436347 - timing_s/adv:0.1540521327406168 - timing_s/update_actor:10.0051255710423 - timing_s/step:38.650182999670506 - timing_s/stop_profile:0.0001270286738872528 - timing_per_token_ms/update_actor:0.04871542646055487 - timing_per_token_ms/gen:0.18738710089602584 - timing_per_token_ms/adv:0.00075008707190422 - perf/total_num_tokens:205379 - perf/time_per_step:38.650182999670506 - perf/throughput:664.2238925548905 (TaskRunner pid=2049544) Training Progress: 67%|██████▋ | 67/100 [51:09<23:14, 42.26s/it] (TaskRunner pid=2049544) step:68 - global_seqlen/min:21420 - global_seqlen/max:36370 - global_seqlen/minmax_diff:14950 - global_seqlen/balanced_min:26998 - global_seqlen/balanced_max:27019 - global_seqlen/mean:27007.25 - actor/entropy:0.2508779466152191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4620983600616455 - training/rollout_probs_diff_mean:0.003281368175521493 - training/rollout_probs_diff_std:0.009971853345632553 - training/rollout_actor_probs_pearson_corr:0.9991183876991272 - rollout_corr/rollout_is_mean:0.9999003410339355 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.37541350745596e-05 - rollout_corr/rollout_is_max:1.727583885192871 - rollout_corr/rollout_is_min:0.3678779602050781 - rollout_corr/rollout_is_std:0.03009769693017006 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:0.999991238117218 - rollout_corr/rollout_is_seq_std:0.0015605590306222439 - rollout_corr/rollout_is_seq_max:1.005854606628418 - rollout_corr/rollout_is_seq_min:0.9957244396209717 - rollout_corr/rollout_is_seq_max_deviation:0.005854606628417969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2427316303364933) - rollout_corr/training_log_ppl:np.float64(0.2052579416122171) - rollout_corr/kl:np.float64(0.0006245356830429927) - rollout_corr/k3_kl:np.float64(0.0004809370777678623) - rollout_corr/rollout_ppl:np.float64(1.2419258821755648) - rollout_corr/rollout_log_ppl:np.float64(0.20463340327114565) - rollout_corr/log_ppl_diff:np.float64(0.0006245383410714567) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013257119280751795) - rollout_corr/log_ppl_diff_max:np.float64(0.005434125661849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038042664527893066) - rollout_corr/ppl_ratio:np.float64(1.0006259840447456) - rollout_corr/chi2_token:np.float64(0.0006594203878194094) - rollout_corr/chi2_seq:np.float64(0.44312575249932706) - actor/pg_loss:np.float64(-5.7110912166535854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002576871158908034) - actor/ppo_kl:np.float64(0.000146509498037517) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1650708168745041) - perf/mfu/actor:np.float64(0.07864794728282518) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82149505615234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015104975318536162 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015104975318536162 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:578.6328125 - response_length/max:1942.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:578.6328125 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.873881936073303e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0787797346711159) - timing_s/agent_loop/generate_sequences/max:np.float64(13.899392530322075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.687431053593173) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.899392530322075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1942 - timing_s/gen:19.621694235131145 - timing_s/reward:0.06307606771588326 - timing_s/old_log_prob:2.414205303415656 - timing_s/adv:0.1221406627446413 - timing_s/update_actor:10.004864446818829 - timing_s/step:32.31770410388708 - timing_s/stop_profile:4.9777328968048096e-05 - timing_per_token_ms/update_actor:0.04630638276212327 - timing_per_token_ms/gen:0.13246266276332375 - timing_per_token_ms/adv:0.0005653142338846111 - perf/total_num_tokens:216058 - perf/time_per_step:32.31770410388708 - perf/throughput:835.6797225812722 (TaskRunner pid=2049544) Training Progress: 68%|██████▊ | 68/100 [51:41<20:57, 39.29s/it] (TaskRunner pid=2049544) step:69 - global_seqlen/min:23146 - global_seqlen/max:36753 - global_seqlen/minmax_diff:13607 - global_seqlen/balanced_min:29343 - global_seqlen/balanced_max:29374 - global_seqlen/mean:29363.125 - actor/entropy:0.2744224965572357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491748213768005 - training/rollout_probs_diff_mean:0.0033163181506097317 - training/rollout_probs_diff_std:0.00957618560642004 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - rollout_corr/rollout_is_mean:0.9999679327011108 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9769123792648315 - rollout_corr/rollout_is_min:0.547244668006897 - rollout_corr/rollout_is_std:0.030115514993667603 - rollout_corr/rollout_is_eff_sample_size:0.9990938775276863 - rollout_corr/rollout_is_seq_mean:1.000019907951355 - rollout_corr/rollout_is_seq_std:0.001533227856270969 - rollout_corr/rollout_is_seq_max:1.0057690143585205 - rollout_corr/rollout_is_seq_min:0.990231990814209 - rollout_corr/rollout_is_seq_max_deviation:0.009768009185791016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2841168828308582) - rollout_corr/training_log_ppl:np.float64(0.23235505366756115) - rollout_corr/kl:np.float64(0.0005125461427646361) - rollout_corr/k3_kl:np.float64(0.0005065118215270559) - rollout_corr/rollout_ppl:np.float64(1.2834058650769293) - rollout_corr/rollout_log_ppl:np.float64(0.2318425054399995) - rollout_corr/log_ppl_diff:np.float64(0.0005125482275616378) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012335704232100397) - rollout_corr/log_ppl_diff_max:np.float64(0.011537343263626099) - rollout_corr/log_ppl_diff_min:np.float64(-0.005053341388702393) - rollout_corr/ppl_ratio:np.float64(1.0005140437278897) - rollout_corr/chi2_token:np.float64(0.00102224200963974) - rollout_corr/chi2_seq:np.float64(0.8634938539471477) - actor/pg_loss:np.float64(-5.3855590522289276e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000181908647164164) - actor/ppo_kl:np.float64(5.1798504737021744e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18398911505937576) - perf/mfu/actor:np.float64(0.08486034559374442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82257843017578) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015627924352884293 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015627924352884293 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:646.87890625 - response_length/max:2353.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.87890625 - response_length_non_aborted/max:2353.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.917755722999573e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.203237522393465) - timing_s/agent_loop/generate_sequences/max:np.float64(16.094254648312926) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.346816844648856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.094254648312926) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:2353 - timing_s/gen:22.094875816255808 - timing_s/reward:0.05950486660003662 - timing_s/old_log_prob:2.4449626710265875 - timing_s/adv:0.12635387293994427 - timing_s/update_actor:10.193446822464466 - timing_s/step:34.9527602866292 - timing_s/stop_profile:2.9163435101509094e-05 - timing_per_token_ms/update_actor:0.04339391167690967 - timing_per_token_ms/gen:0.13342235745107703 - timing_per_token_ms/adv:0.0005378935013726581 - perf/total_num_tokens:234905 - perf/time_per_step:34.9527602866292 - perf/throughput:840.0802900603117 (TaskRunner pid=2049544) Training Progress: 69%|██████▉ | 69/100 [52:16<19:37, 37.99s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2049544) step:70 - global_seqlen/min:18463 - global_seqlen/max:39933 - global_seqlen/minmax_diff:21470 - global_seqlen/balanced_min:27119 - global_seqlen/balanced_max:27177 - global_seqlen/mean:27165.625 - actor/entropy:0.34878426790237427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35282185673713684 - training/rollout_probs_diff_mean:0.003718312829732895 - training/rollout_probs_diff_std:0.009631498716771603 - training/rollout_actor_probs_pearson_corr:0.999345600605011 - rollout_corr/rollout_is_mean:1.000012993812561 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0160612621111795e-05 - rollout_corr/rollout_is_max:1.975199818611145 - rollout_corr/rollout_is_min:0.19730398058891296 - rollout_corr/rollout_is_std:0.03194605931639671 - rollout_corr/rollout_is_eff_sample_size:0.9989806087004571 - rollout_corr/rollout_is_seq_mean:1.0000489950180054 - rollout_corr/rollout_is_seq_std:0.0017589139752089977 - rollout_corr/rollout_is_seq_max:1.0087381601333618 - rollout_corr/rollout_is_seq_min:0.992647647857666 - rollout_corr/rollout_is_seq_max_deviation:0.008738160133361816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3483932986855507) - rollout_corr/training_log_ppl:np.float64(0.2762820063435356) - rollout_corr/kl:np.float64(0.0006076907076355553) - rollout_corr/k3_kl:np.float64(0.0005228528713416836) - rollout_corr/rollout_ppl:np.float64(1.347519775852561) - rollout_corr/rollout_log_ppl:np.float64(0.27567431312490953) - rollout_corr/log_ppl_diff:np.float64(0.0006076932186260819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012910107034258544) - rollout_corr/log_ppl_diff_max:np.float64(0.009554415941238403) - rollout_corr/log_ppl_diff_min:np.float64(-0.007596790790557861) - rollout_corr/ppl_ratio:np.float64(1.0006093357224017) - rollout_corr/chi2_token:np.float64(0.0008673274423927069) - rollout_corr/chi2_seq:np.float64(0.4545943159610033) - actor/pg_loss:np.float64(4.701910074800253e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021670004468887782) - actor/ppo_kl:np.float64(0.0001372517156355002) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2546451985836029) - perf/mfu/actor:np.float64(0.07849234372480743) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.89488983154297) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/score/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/score/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06817587534083876) - val-core/sciknoweval/acc/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09773915943586196) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8506) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07051862491369472) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6758124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.041294709008662854) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9961125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.016882665811088163) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.966925) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04991881605672479) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9813124999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04115652748695172) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004110435910939457) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9449500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05482807368910221) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9914250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.027274560942092924) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.92125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05037233272292474) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.01160302730045884) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8982125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.037001893862708676) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9991249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.005012406355781736) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005754174781031907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005754174781031907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:581.26953125 - response_length/max:2622.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:581.26953125 - response_length_non_aborted/max:2622.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7524307370185852e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.118723189458251) - timing_s/agent_loop/generate_sequences/max:np.float64(16.3854166790843) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.583953251349158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.3854166790843) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2622 - timing_s/gen:22.14735895767808 - timing_s/reward:0.0681693684309721 - timing_s/old_log_prob:2.4311879333108664 - timing_s/adv:0.11663812026381493 - timing_s/update_actor:9.943649336695671 - timing_s/step:34.796333791688085 - timing_s/testing:115.27732796221972 - timing_s/save_checkpoint:6.6401550360023975 - timing_s/stop_profile:0.000131901353597641 - timing_per_token_ms/update_actor:0.04575474214515436 - timing_per_token_ms/gen:0.1488347767728106 - timing_per_token_ms/adv:0.0005366990464227075 - perf/total_num_tokens:217325 - perf/time_per_step:34.796333791688085 - perf/throughput:780.7036558112666 (TaskRunner pid=2049544) Training Progress: 70%|███████ | 70/100 [54:53<36:48, 73.62s/it] (TaskRunner pid=2049544) step:71 - global_seqlen/min:20204 - global_seqlen/max:38376 - global_seqlen/minmax_diff:18172 - global_seqlen/balanced_min:28759 - global_seqlen/balanced_max:29243 - global_seqlen/mean:28856.5 - actor/entropy:0.32371971011161804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35350361466407776 - training/rollout_probs_diff_mean:0.003485669381916523 - training/rollout_probs_diff_std:0.009408628568053246 - training/rollout_actor_probs_pearson_corr:0.9993824362754822 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:6.257665518205613e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.257665518205613e-06 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.4736731946468353 - rollout_corr/rollout_is_std:0.03072100132703781 - rollout_corr/rollout_is_eff_sample_size:0.9990571099353601 - rollout_corr/rollout_is_seq_mean:1.0000032186508179 - rollout_corr/rollout_is_seq_std:0.0015029723290354013 - rollout_corr/rollout_is_seq_max:1.0061728954315186 - rollout_corr/rollout_is_seq_min:0.9937267303466797 - rollout_corr/rollout_is_seq_max_deviation:0.0062732696533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3052765680477023) - rollout_corr/training_log_ppl:np.float64(0.24597317739971913) - rollout_corr/kl:np.float64(0.00043000688371730433) - rollout_corr/k3_kl:np.float64(0.000540082649621354) - rollout_corr/rollout_ppl:np.float64(1.3047043732367456) - rollout_corr/rollout_log_ppl:np.float64(0.2455431708949618) - rollout_corr/log_ppl_diff:np.float64(0.0004300065047573298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012972943659406155) - rollout_corr/log_ppl_diff_max:np.float64(0.00797504186630249) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062088742852211) - rollout_corr/ppl_ratio:np.float64(1.0004315802361816) - rollout_corr/chi2_token:np.float64(0.0013474705629050732) - rollout_corr/chi2_seq:np.float64(1.4930782679002732) - actor/pg_loss:np.float64(0.0001349709928035736) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038389957057916035) - actor/ppo_kl:np.float64(-3.772802561208266e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17576251924037933) - perf/mfu/actor:np.float64(0.08244675822096596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2421646118164) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009262127801775932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009262127801775932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:624.234375 - response_length/max:3404.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:624.234375 - response_length_non_aborted/max:3404.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012946873903274536 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2765609622001648) - timing_s/agent_loop/generate_sequences/max:np.float64(20.10214041545987) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.958212114004709) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.10214041545987) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:3404 - timing_s/gen:25.802896425127983 - timing_s/reward:0.07092655450105667 - timing_s/old_log_prob:2.582106838002801 - timing_s/adv:0.1316581778228283 - timing_s/update_actor:10.102314312011003 - timing_s/step:38.781780099496245 - timing_s/stop_profile:0.00012562982738018036 - timing_per_token_ms/update_actor:0.04376099974014089 - timing_per_token_ms/gen:0.16146589838256853 - timing_per_token_ms/adv:0.0005703142178661146 - perf/total_num_tokens:230852 - perf/time_per_step:38.781780099496245 - perf/throughput:744.0736326689355 (TaskRunner pid=2049544) Training Progress: 71%|███████ | 71/100 [55:32<30:32, 63.19s/it] (TaskRunner pid=2049544) step:72 - global_seqlen/min:24019 - global_seqlen/max:35372 - global_seqlen/minmax_diff:11353 - global_seqlen/balanced_min:30681 - global_seqlen/balanced_max:31067 - global_seqlen/mean:30756.75 - actor/entropy:0.2652339041233063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5334677696228027 - training/rollout_probs_diff_mean:0.003048401791602373 - training/rollout_probs_diff_std:0.009212449193000793 - training/rollout_actor_probs_pearson_corr:0.9992727637290955 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:1.1159842870256398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.6739762941142544e-05 - rollout_corr/rollout_is_max:2.703545570373535 - rollout_corr/rollout_is_min:0.2786954641342163 - rollout_corr/rollout_is_std:0.02862177975475788 - rollout_corr/rollout_is_eff_sample_size:0.9991814643117487 - rollout_corr/rollout_is_seq_mean:1.0000512599945068 - rollout_corr/rollout_is_seq_std:0.0012803305871784687 - rollout_corr/rollout_is_seq_max:1.005553960800171 - rollout_corr/rollout_is_seq_min:0.9965124726295471 - rollout_corr/rollout_is_seq_max_deviation:0.0055539608001708984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.272761202417314) - rollout_corr/training_log_ppl:np.float64(0.2287760855979286) - rollout_corr/kl:np.float64(0.0003352667156821809) - rollout_corr/k3_kl:np.float64(0.0004365081685619998) - rollout_corr/rollout_ppl:np.float64(1.2723114336840808) - rollout_corr/rollout_log_ppl:np.float64(0.22844081881339662) - rollout_corr/log_ppl_diff:np.float64(0.00033526678453199565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010565653501544148) - rollout_corr/log_ppl_diff_max:np.float64(0.00969955325126648) - rollout_corr/log_ppl_diff_min:np.float64(-0.005003869533538818) - rollout_corr/ppl_ratio:np.float64(1.0003363261930645) - rollout_corr/chi2_token:np.float64(0.0010922125075012445) - rollout_corr/chi2_seq:np.float64(1.312244726344943) - actor/pg_loss:np.float64(8.467817679047585e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000129777238043971) - actor/ppo_kl:np.float64(-3.5065210759910315e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1640554666519165) - perf/mfu/actor:np.float64(0.08815191325739583) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.06222915649414) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006141400896012783 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006141400896012783 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:700.0546875 - response_length/max:3313.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:700.0546875 - response_length_non_aborted/max:3313.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013216771185398102 - timing_s/agent_loop/generate_sequences/min:np.float64(2.07568240724504) - timing_s/agent_loop/generate_sequences/max:np.float64(20.67868852801621) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.946097736770753) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.67868852801621) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3313 - timing_s/gen:26.445967487990856 - timing_s/reward:0.0720598753541708 - timing_s/old_log_prob:2.519180826842785 - timing_s/adv:0.12123813852667809 - timing_s/update_actor:10.210091643035412 - timing_s/step:39.45813396573067 - timing_s/stop_profile:0.00012371689081192017 - timing_per_token_ms/update_actor:0.04149532884259314 - timing_per_token_ms/gen:0.14756641494520994 - timing_per_token_ms/adv:0.0004927298012902781 - perf/total_num_tokens:246054 - perf/time_per_step:39.45813396573067 - perf/throughput:779.4780672272083 (TaskRunner pid=2049544) Training Progress: 72%|███████▏ | 72/100 [56:11<26:10, 56.08s/it] (TaskRunner pid=2049544) step:73 - global_seqlen/min:21925 - global_seqlen/max:46347 - global_seqlen/minmax_diff:24422 - global_seqlen/balanced_min:36563 - global_seqlen/balanced_max:36628 - global_seqlen/mean:36610.375 - actor/entropy:0.3390262722969055 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43396425247192383 - training/rollout_probs_diff_mean:0.0032661668956279755 - training/rollout_probs_diff_std:0.008885001763701439 - training/rollout_actor_probs_pearson_corr:0.9994308948516846 - rollout_corr/rollout_is_mean:0.9999864101409912 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7025687813875265e-05 - rollout_corr/rollout_is_max:1.9699597358703613 - rollout_corr/rollout_is_min:0.2820616662502289 - rollout_corr/rollout_is_std:0.029777150601148605 - rollout_corr/rollout_is_eff_sample_size:0.9991141068069422 - rollout_corr/rollout_is_seq_mean:0.9999426007270813 - rollout_corr/rollout_is_seq_std:0.0015595207223668694 - rollout_corr/rollout_is_seq_max:1.0064419507980347 - rollout_corr/rollout_is_seq_min:0.992620587348938 - rollout_corr/rollout_is_seq_max_deviation:0.007379412651062012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4027120932005346) - rollout_corr/training_log_ppl:np.float64(0.31371801071509253) - rollout_corr/kl:np.float64(0.000663529713321509) - rollout_corr/k3_kl:np.float64(0.0005334890627182176) - rollout_corr/rollout_ppl:np.float64(1.4016525098122656) - rollout_corr/rollout_log_ppl:np.float64(0.3130544779851334) - rollout_corr/log_ppl_diff:np.float64(0.0006635327299591154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013385674392338842) - rollout_corr/log_ppl_diff_max:np.float64(0.008049964904785156) - rollout_corr/log_ppl_diff_min:np.float64(-0.005723848938941956) - rollout_corr/ppl_ratio:np.float64(1.0006653110031039) - rollout_corr/chi2_token:np.float64(0.0007862187922000885) - rollout_corr/chi2_seq:np.float64(0.7408852751832455) - actor/pg_loss:np.float64(-9.118067100644112e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025507958866910485) - actor/ppo_kl:np.float64(9.864521248204028e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22031916305422783) - perf/mfu/actor:np.float64(0.10239718785831436) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18194198608398) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006957425735890865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006957425735890865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:867.23046875 - response_length/max:3153.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:867.23046875 - response_length_non_aborted/max:3153.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944902896881104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9543074388056993) - timing_s/agent_loop/generate_sequences/max:np.float64(21.861840976402164) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.284751043880533) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.861840976402164) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3153 - timing_s/gen:27.61041198298335 - timing_s/reward:0.07980250380933285 - timing_s/old_log_prob:2.6316978093236685 - timing_s/adv:0.14758859016001225 - timing_s/update_actor:10.437388250604272 - timing_s/step:40.99476144276559 - timing_s/stop_profile:0.00011257268488407135 - timing_per_token_ms/update_actor:0.03563671585788274 - timing_per_token_ms/gen:0.12436506291572648 - timing_per_token_ms/adv:0.00050391654742683 - perf/total_num_tokens:292883 - perf/time_per_step:40.99476144276559 - perf/throughput:893.0500803404648 (TaskRunner pid=2049544) Training Progress: 73%|███████▎ | 73/100 [56:52<23:12, 51.57s/it] (TaskRunner pid=2049544) step:74 - global_seqlen/min:18401 - global_seqlen/max:45796 - global_seqlen/minmax_diff:27395 - global_seqlen/balanced_min:30702 - global_seqlen/balanced_max:30981 - global_seqlen/mean:30746.0 - actor/entropy:0.31715160608291626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399150013923645 - training/rollout_probs_diff_mean:0.0034521878696978092 - training/rollout_probs_diff_std:0.009627916850149632 - training/rollout_actor_probs_pearson_corr:0.9993271231651306 - rollout_corr/rollout_is_mean:1.0001097917556763 - rollout_corr/rollout_is_ratio_fraction_high:1.1016612916137092e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.652492028370034e-05 - rollout_corr/rollout_is_max:2.2787067890167236 - rollout_corr/rollout_is_min:0.41993895173072815 - rollout_corr/rollout_is_std:0.030713239684700966 - rollout_corr/rollout_is_eff_sample_size:0.9990577048586795 - rollout_corr/rollout_is_seq_mean:1.000206708908081 - rollout_corr/rollout_is_seq_std:0.002247885102406144 - rollout_corr/rollout_is_seq_max:1.0164759159088135 - rollout_corr/rollout_is_seq_min:0.9938144683837891 - rollout_corr/rollout_is_seq_max_deviation:0.016475915908813477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3208666159771383) - rollout_corr/training_log_ppl:np.float64(0.2606007744325325) - rollout_corr/kl:np.float64(0.0003702222969650393) - rollout_corr/k3_kl:np.float64(0.000720645587279023) - rollout_corr/rollout_ppl:np.float64(1.3202885040082037) - rollout_corr/rollout_log_ppl:np.float64(0.26023055279802065) - rollout_corr/log_ppl_diff:np.float64(0.0003702216345118359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016211537149501964) - rollout_corr/log_ppl_diff_max:np.float64(0.01012013852596283) - rollout_corr/log_ppl_diff_min:np.float64(-0.012066826224327087) - rollout_corr/ppl_ratio:np.float64(1.0003729804884642) - rollout_corr/chi2_token:np.float64(0.0021484598983079195) - rollout_corr/chi2_seq:np.float64(2.6817988054826856) - actor/pg_loss:np.float64(-0.00016311672516167164) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024571236758674786) - actor/ppo_kl:np.float64(-5.575577448624358e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21021737158298492) - perf/mfu/actor:np.float64(0.08564326289574913) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.05369186401367) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008819486945867538 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008819486945867538 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:709.15625 - response_length/max:4610.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:709.15625 - response_length_non_aborted/max:4610.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.643472731113434e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46839356422424316) - timing_s/agent_loop/generate_sequences/max:np.float64(26.743163952603936) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.783247412800847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.743163952603936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:4610 - timing_s/gen:33.11821131967008 - timing_s/reward:0.07101857103407383 - timing_s/old_log_prob:2.5942629985511303 - timing_s/adv:0.12228076718747616 - timing_s/update_actor:10.489771209657192 - timing_s/step:46.487874053418636 - timing_s/stop_profile:0.00010906346142292023 - timing_per_token_ms/update_actor:0.042646893944160186 - timing_per_token_ms/gen:0.18242525954958622 - timing_per_token_ms/adv:0.0004971409581225044 - perf/total_num_tokens:245968 - perf/time_per_step:46.487874053418636 - perf/throughput:661.3767703093963 (TaskRunner pid=2049544) Training Progress: 74%|███████▍ | 74/100 [57:39<21:41, 50.06s/it] (TaskRunner pid=2049544) step:75 - global_seqlen/min:19402 - global_seqlen/max:28818 - global_seqlen/minmax_diff:9416 - global_seqlen/balanced_min:23785 - global_seqlen/balanced_max:23996 - global_seqlen/mean:23814.375 - actor/entropy:0.25116294622421265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135130643844604 - training/rollout_probs_diff_mean:0.0031402192544192076 - training/rollout_probs_diff_std:0.009609016589820385 - training/rollout_actor_probs_pearson_corr:0.9991532564163208 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:8.058602361415979e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4175806174753234e-05 - rollout_corr/rollout_is_max:2.1670117378234863 - rollout_corr/rollout_is_min:0.4087337255477905 - rollout_corr/rollout_is_std:0.03037172183394432 - rollout_corr/rollout_is_eff_sample_size:0.9990784086315643 - rollout_corr/rollout_is_seq_mean:1.0000663995742798 - rollout_corr/rollout_is_seq_std:0.0019097818294540048 - rollout_corr/rollout_is_seq_max:1.0100950002670288 - rollout_corr/rollout_is_seq_min:0.9921054244041443 - rollout_corr/rollout_is_seq_max_deviation:0.010095000267028809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2536835842765868) - rollout_corr/training_log_ppl:np.float64(0.20439182232803432) - rollout_corr/kl:np.float64(0.00044191136634275097) - rollout_corr/k3_kl:np.float64(0.0004956063722261206) - rollout_corr/rollout_ppl:np.float64(1.2531235865317285) - rollout_corr/rollout_log_ppl:np.float64(0.20394990891509224) - rollout_corr/log_ppl_diff:np.float64(0.0004419134129420854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013838859085808508) - rollout_corr/log_ppl_diff_max:np.float64(0.009681947529315948) - rollout_corr/log_ppl_diff_min:np.float64(-0.005931958556175232) - rollout_corr/ppl_ratio:np.float64(1.0004437156021595) - rollout_corr/chi2_token:np.float64(0.0011102964635938406) - rollout_corr/chi2_seq:np.float64(0.39504921343177557) - actor/pg_loss:np.float64(1.7428305000066757e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018432266847412393) - actor/ppo_kl:np.float64(3.7105337884923983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16733924113214016) - perf/mfu/actor:np.float64(0.06918613903772251) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.97526931762695) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008511898340657353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008511898340657353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:484.73046875 - response_length/max:2220.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.73046875 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016090832650661469 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2748903073370457) - timing_s/agent_loop/generate_sequences/max:np.float64(13.663885425776243) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.775116956734564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.663885425776243) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:19.10645161010325 - timing_s/reward:0.06184021010994911 - timing_s/old_log_prob:2.374053070321679 - timing_s/adv:0.11829666048288345 - timing_s/update_actor:9.632479302585125 - timing_s/step:31.360355062410235 - timing_s/stop_profile:3.0426308512687683e-05 - timing_per_token_ms/update_actor:0.05056021469482783 - timing_per_token_ms/gen:0.15397129211710156 - timing_per_token_ms/adv:0.0006209309528534942 - perf/total_num_tokens:190515 - perf/time_per_step:31.360355062410235 - perf/throughput:759.3783601176395 (TaskRunner pid=2049544) Training Progress: 75%|███████▌ | 75/100 [58:10<18:31, 44.46s/it] (TaskRunner pid=2049544) step:76 - global_seqlen/min:25932 - global_seqlen/max:42011 - global_seqlen/minmax_diff:16079 - global_seqlen/balanced_min:31987 - global_seqlen/balanced_max:32015 - global_seqlen/mean:32005.0 - actor/entropy:0.35323843359947205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8143891096115112 - training/rollout_probs_diff_mean:0.003405920695513487 - training/rollout_probs_diff_std:0.009273294359445572 - training/rollout_actor_probs_pearson_corr:0.9994474053382874 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:1.081595564755844e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.081595564755844e-05 - rollout_corr/rollout_is_max:3.5232980251312256 - rollout_corr/rollout_is_min:0.16788354516029358 - rollout_corr/rollout_is_std:0.030408985912799835 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:1.0001401901245117 - rollout_corr/rollout_is_seq_std:0.0016239474061876535 - rollout_corr/rollout_is_seq_max:1.010636329650879 - rollout_corr/rollout_is_seq_min:0.9931896328926086 - rollout_corr/rollout_is_seq_max_deviation:0.010636329650878906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3853302095085382) - rollout_corr/training_log_ppl:np.float64(0.3024687277356861) - rollout_corr/kl:np.float64(0.0005689695157187202) - rollout_corr/k3_kl:np.float64(0.0005752426737899441) - rollout_corr/rollout_ppl:np.float64(1.38448647223413) - rollout_corr/rollout_log_ppl:np.float64(0.30189975767279975) - rollout_corr/log_ppl_diff:np.float64(0.0005689700628863648) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012543359625851735) - rollout_corr/log_ppl_diff_max:np.float64(0.007834061980247498) - rollout_corr/log_ppl_diff_min:np.float64(-0.006428062915802002) - rollout_corr/ppl_ratio:np.float64(1.0005704772192985) - rollout_corr/chi2_token:np.float64(0.0012044101022183895) - rollout_corr/chi2_seq:np.float64(1.3872064049355686) - actor/pg_loss:np.float64(5.75545709580183e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002793574865336268) - actor/ppo_kl:np.float64(0.00019198946114684645) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3208663538098335) - perf/mfu/actor:np.float64(0.09134199443488283) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.03634643554688) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008177538402378559 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008177538402378559 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:722.3125 - response_length/max:2396.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:722.3125 - response_length_non_aborted/max:2396.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.119371831417084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.99646220728755) - timing_s/agent_loop/generate_sequences/max:np.float64(16.732603766024113) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.14587117084011) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.732603766024113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:2371 - timing_s/gen:22.66927962191403 - timing_s/reward:0.07421145401895046 - timing_s/old_log_prob:2.5292600374668837 - timing_s/adv:0.1157302763313055 - timing_s/update_actor:10.305034121498466 - timing_s/step:35.780901934951544 - timing_s/stop_profile:0.00012325309216976166 - timing_per_token_ms/update_actor:0.040247750826036814 - timing_per_token_ms/gen:0.12259496204634653 - timing_per_token_ms/adv:0.00045200076679934975 - perf/total_num_tokens:256040 - perf/time_per_step:35.780901934951544 - perf/throughput:894.471583141867 (TaskRunner pid=2049544) Training Progress: 76%|███████▌ | 76/100 [58:46<16:44, 41.87s/it] (TaskRunner pid=2049544) step:77 - global_seqlen/min:16244 - global_seqlen/max:41465 - global_seqlen/minmax_diff:25221 - global_seqlen/balanced_min:29420 - global_seqlen/balanced_max:29517 - global_seqlen/mean:29492.5 - actor/entropy:0.3045775294303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915552258491516 - training/rollout_probs_diff_mean:0.0032992891501635313 - training/rollout_probs_diff_std:0.009267176501452923 - training/rollout_actor_probs_pearson_corr:0.9993472695350647 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.8924742916133255e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.140592098236084 - rollout_corr/rollout_is_min:0.5226767659187317 - rollout_corr/rollout_is_std:0.029582349583506584 - rollout_corr/rollout_is_eff_sample_size:0.9991255307618554 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.0017432396998628974 - rollout_corr/rollout_is_seq_max:1.008344054222107 - rollout_corr/rollout_is_seq_min:0.9936603307723999 - rollout_corr/rollout_is_seq_max_deviation:0.008344054222106934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2967352382838726) - rollout_corr/training_log_ppl:np.float64(0.24739519167633262) - rollout_corr/kl:np.float64(0.0005334063558146696) - rollout_corr/k3_kl:np.float64(0.0005630067024355867) - rollout_corr/rollout_ppl:np.float64(1.2960236202925444) - rollout_corr/rollout_log_ppl:np.float64(0.24686178589763585) - rollout_corr/log_ppl_diff:np.float64(0.0005334057786967605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014061912952456623) - rollout_corr/log_ppl_diff_max:np.float64(0.01065683364868164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076304227113723755) - rollout_corr/ppl_ratio:np.float64(1.0005355728790164) - rollout_corr/chi2_token:np.float64(0.0011709912214428186) - rollout_corr/chi2_seq:np.float64(0.44916587066836655) - actor/pg_loss:np.float64(2.6725465431809425e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018346831632243266) - actor/ppo_kl:np.float64(9.181251817746983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16028263419866562) - perf/mfu/actor:np.float64(0.08482457572663438) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.09542083740234) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006819802336394787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006819802336394787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:662.921875 - response_length/max:2972.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:662.921875 - response_length_non_aborted/max:2972.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013144128024578094 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8779960200190544) - timing_s/agent_loop/generate_sequences/max:np.float64(18.480581056326628) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.09187524789013) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.480581056326628) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2972 - timing_s/gen:24.529541220515966 - timing_s/reward:0.07079425640404224 - timing_s/old_log_prob:2.487271063029766 - timing_s/adv:0.11753782816231251 - timing_s/update_actor:10.21386044844985 - timing_s/step:37.50457299686968 - timing_s/stop_profile:2.83215194940567e-05 - timing_per_token_ms/update_actor:0.043290075648257396 - timing_per_token_ms/gen:0.1445396871126639 - timing_per_token_ms/adv:0.0004981682977126071 - perf/total_num_tokens:235940 - perf/time_per_step:37.50457299686968 - perf/throughput:786.3707714379681 (TaskRunner pid=2049544) Training Progress: 77%|███████▋ | 77/100 [59:24<15:33, 40.57s/it] (TaskRunner pid=2049544) step:78 - global_seqlen/min:16539 - global_seqlen/max:34942 - global_seqlen/minmax_diff:18403 - global_seqlen/balanced_min:26643 - global_seqlen/balanced_max:26991 - global_seqlen/mean:26772.625 - actor/entropy:0.324239581823349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30065640807151794 - training/rollout_probs_diff_mean:0.003476528450846672 - training/rollout_probs_diff_std:0.009609517641365528 - training/rollout_actor_probs_pearson_corr:0.9993136525154114 - rollout_corr/rollout_is_mean:0.9999270439147949 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.365252956020413e-05 - rollout_corr/rollout_is_max:1.927759051322937 - rollout_corr/rollout_is_min:0.17178574204444885 - rollout_corr/rollout_is_std:0.031049082055687904 - rollout_corr/rollout_is_eff_sample_size:0.9990367639843101 - rollout_corr/rollout_is_seq_mean:0.9997465014457703 - rollout_corr/rollout_is_seq_std:0.0017120024422183633 - rollout_corr/rollout_is_seq_max:1.0067331790924072 - rollout_corr/rollout_is_seq_min:0.9950670599937439 - rollout_corr/rollout_is_seq_max_deviation:0.0067331790924072266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026048908941448) - rollout_corr/training_log_ppl:np.float64(0.24134253649390303) - rollout_corr/kl:np.float64(0.0007427987794557112) - rollout_corr/k3_kl:np.float64(0.0005388006466660045) - rollout_corr/rollout_ppl:np.float64(1.301613979972899) - rollout_corr/rollout_log_ppl:np.float64(0.24059973818657454) - rollout_corr/log_ppl_diff:np.float64(0.000742798307328485) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014510368491755798) - rollout_corr/log_ppl_diff_max:np.float64(0.0068367719650268555) - rollout_corr/log_ppl_diff_min:np.float64(-0.005617767572402954) - rollout_corr/ppl_ratio:np.float64(1.0007447020616382) - rollout_corr/chi2_token:np.float64(0.0006613847799599171) - rollout_corr/chi2_seq:np.float64(0.6144026103429496) - actor/pg_loss:np.float64(5.164649337530136e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011791538452143868) - actor/ppo_kl:np.float64(-3.060101226282086e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16195594891905785) - perf/mfu/actor:np.float64(0.07693655141257658) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.07821655273438) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012876042164862156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012876042164862156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:572.23828125 - response_length/max:3226.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:572.23828125 - response_length_non_aborted/max:3226.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.397596836090088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2444690354168415) - timing_s/agent_loop/generate_sequences/max:np.float64(19.064789429306984) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.27938631172583) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.064789429306984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:3226 - timing_s/gen:24.83233668655157 - timing_s/reward:0.06715011782944202 - timing_s/old_log_prob:2.454452695325017 - timing_s/adv:0.12591312639415264 - timing_s/update_actor:10.206827508285642 - timing_s/step:37.77338271215558 - timing_s/stop_profile:0.00012371130287647247 - timing_per_token_ms/update_actor:0.04765514918823631 - timing_per_token_ms/gen:0.1695121042408277 - timing_per_token_ms/adv:0.0005878818681122632 - perf/total_num_tokens:214181 - perf/time_per_step:37.77338271215558 - perf/throughput:708.7695905875142 (TaskRunner pid=2049544) Training Progress: 78%|███████▊ | 78/100 [1:00:02<14:34, 39.74s/it] (TaskRunner pid=2049544) step:79 - global_seqlen/min:20249 - global_seqlen/max:46165 - global_seqlen/minmax_diff:25916 - global_seqlen/balanced_min:30262 - global_seqlen/balanced_max:30358 - global_seqlen/mean:30336.375 - actor/entropy:0.27736997604370117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4035380482673645 - training/rollout_probs_diff_mean:0.0030377230141311884 - training/rollout_probs_diff_std:0.009076771326363087 - training/rollout_actor_probs_pearson_corr:0.9993468523025513 - rollout_corr/rollout_is_mean:0.9998963475227356 - rollout_corr/rollout_is_ratio_fraction_high:5.790153409179766e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.211137977312319e-05 - rollout_corr/rollout_is_max:4.029329299926758 - rollout_corr/rollout_is_min:0.07581690698862076 - rollout_corr/rollout_is_std:0.02898186817765236 - rollout_corr/rollout_is_eff_sample_size:0.9991605182489611 - rollout_corr/rollout_is_seq_mean:0.9999415874481201 - rollout_corr/rollout_is_seq_std:0.0017486511496827006 - rollout_corr/rollout_is_seq_max:1.0085891485214233 - rollout_corr/rollout_is_seq_min:0.9941185712814331 - rollout_corr/rollout_is_seq_max_deviation:0.00858914852142334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2839201726019382) - rollout_corr/training_log_ppl:np.float64(0.2279481466684956) - rollout_corr/kl:np.float64(0.0005260294094213691) - rollout_corr/k3_kl:np.float64(0.0005059725925065095) - rollout_corr/rollout_ppl:np.float64(1.2831925540231168) - rollout_corr/rollout_log_ppl:np.float64(0.2274221143743489) - rollout_corr/log_ppl_diff:np.float64(0.0005260322941467166) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012799371324945241) - rollout_corr/log_ppl_diff_max:np.float64(0.008957505226135254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052579790353775024) - rollout_corr/ppl_ratio:np.float64(1.0005276040174067) - rollout_corr/chi2_token:np.float64(0.0009765762370079756) - rollout_corr/chi2_seq:np.float64(0.7638932466506958) - actor/pg_loss:np.float64(0.00014414661563932896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017305412563928257) - actor/ppo_kl:np.float64(-2.993323226707645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20632774755358696) - perf/mfu/actor:np.float64(0.08763345813710535) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.01940155029297) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008221293799579144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008221293799579144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:674.63671875 - response_length/max:2725.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:674.63671875 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010620616376399994 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1851273197680712) - timing_s/agent_loop/generate_sequences/max:np.float64(17.865634333342314) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.507378077250905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.865634333342314) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2725 - timing_s/gen:23.788950365036726 - timing_s/reward:0.07290799915790558 - timing_s/old_log_prob:2.4474152121692896 - timing_s/adv:0.1101104523986578 - timing_s/update_actor:10.210453910753131 - timing_s/step:36.71885127387941 - timing_s/stop_profile:0.00012296251952648163 - timing_per_token_ms/update_actor:0.04207182759456729 - timing_per_token_ms/gen:0.13774166863553142 - timing_per_token_ms/adv:0.0004537063689986765 - perf/total_num_tokens:242691 - perf/time_per_step:36.71885127387941 - perf/throughput:826.1798489752948 (TaskRunner pid=2049544) Training Progress: 79%|███████▉ | 79/100 [1:00:38<13:35, 38.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2049544) step:80 - global_seqlen/min:18167 - global_seqlen/max:36780 - global_seqlen/minmax_diff:18613 - global_seqlen/balanced_min:28127 - global_seqlen/balanced_max:28499 - global_seqlen/mean:28202.25 - actor/entropy:0.23095937073230743 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44294559955596924 - training/rollout_probs_diff_mean:0.002918842015787959 - training/rollout_probs_diff_std:0.0094361687079072 - training/rollout_actor_probs_pearson_corr:0.9991447329521179 - rollout_corr/rollout_is_mean:0.9999209046363831 - rollout_corr/rollout_is_ratio_fraction_high:1.9175455236108974e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9175455236108974e-05 - rollout_corr/rollout_is_max:2.320603370666504 - rollout_corr/rollout_is_min:0.33252665400505066 - rollout_corr/rollout_is_std:0.028494464233517647 - rollout_corr/rollout_is_eff_sample_size:0.999188605215701 - rollout_corr/rollout_is_seq_mean:0.9999192953109741 - rollout_corr/rollout_is_seq_std:0.001614381093531847 - rollout_corr/rollout_is_seq_max:1.0071684122085571 - rollout_corr/rollout_is_seq_min:0.9931057691574097 - rollout_corr/rollout_is_seq_max_deviation:0.007168412208557129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2288768324069679) - rollout_corr/training_log_ppl:np.float64(0.19351409275259357) - rollout_corr/kl:np.float64(0.0005485934744235976) - rollout_corr/k3_kl:np.float64(0.0005014402207166313) - rollout_corr/rollout_ppl:np.float64(1.2281699823215604) - rollout_corr/rollout_log_ppl:np.float64(0.1929654997366015) - rollout_corr/log_ppl_diff:np.float64(0.0005485930159920827) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800928816432133) - rollout_corr/log_ppl_diff_max:np.float64(0.00843721628189087) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566546857357025) - rollout_corr/ppl_ratio:np.float64(1.0005502616986632) - rollout_corr/chi2_token:np.float64(0.0008651481475681067) - rollout_corr/chi2_seq:np.float64(0.5386446032207459) - actor/pg_loss:np.float64(-2.616771962493658e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001737377756398928) - actor/ppo_kl:np.float64(2.498516215898583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1959109902381897) - perf/mfu/actor:np.float64(0.08114905418503308) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.12442779541016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/score/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/score/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05843863672565752) - val-core/sciknoweval/acc/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10302193213109104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8416625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04059727090755087) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.683725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.06354165489638078) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.023966893234937735) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9539) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08044933488834552) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9746749999999998) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06396939198866787) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998125000000002) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003145810047315943) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9212250000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.09233866134852486) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9907999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03956667018273212) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8777874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08983250151225204) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9978250000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.018387073504573324) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8370125000000002) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0672232524014538) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004527947017243354) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006327900337055326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006327900337055326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:611.1328125 - response_length/max:2845.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:611.1328125 - response_length_non_aborted/max:2845.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000109061598777771 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1131607126444578) - timing_s/agent_loop/generate_sequences/max:np.float64(17.91330724582076) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.142933668168553) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.91330724582076) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:2845 - timing_s/gen:23.700068602338433 - timing_s/reward:0.06908616237342358 - timing_s/old_log_prob:2.398565139621496 - timing_s/adv:0.14442053809762 - timing_s/update_actor:10.117256434634328 - timing_s/step:36.51560699753463 - timing_s/testing:115.6058545038104 - timing_s/save_checkpoint:6.825648196041584 - timing_s/stop_profile:0.00023227371275424957 - timing_per_token_ms/update_actor:0.04484241698195325 - timing_per_token_ms/gen:0.15148653628851666 - timing_per_token_ms/adv:0.0006401108869754187 - perf/total_num_tokens:225618 - perf/time_per_step:36.51560699753463 - perf/throughput:772.3341419986278 (TaskRunner pid=2049544) Training Progress: 80%|████████ | 80/100 [1:03:17<24:57, 74.90s/it] (TaskRunner pid=2049544) step:81 - global_seqlen/min:22002 - global_seqlen/max:45026 - global_seqlen/minmax_diff:23024 - global_seqlen/balanced_min:31986 - global_seqlen/balanced_max:32078 - global_seqlen/mean:32047.5 - actor/entropy:0.30386513471603394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135500192642212 - training/rollout_probs_diff_mean:0.003210825379937887 - training/rollout_probs_diff_std:0.00900829304009676 - training/rollout_actor_probs_pearson_corr:0.9993411898612976 - rollout_corr/rollout_is_mean:1.0000028610229492 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263490493234713e-06 - rollout_corr/rollout_is_max:1.9500842094421387 - rollout_corr/rollout_is_min:0.46151214838027954 - rollout_corr/rollout_is_std:0.029131613671779633 - rollout_corr/rollout_is_eff_sample_size:0.9991520686654056 - rollout_corr/rollout_is_seq_mean:0.9999109506607056 - rollout_corr/rollout_is_seq_std:0.0019725700840353966 - rollout_corr/rollout_is_seq_max:1.0081653594970703 - rollout_corr/rollout_is_seq_min:0.9903681874275208 - rollout_corr/rollout_is_seq_max_deviation:0.009631812572479248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3244363577105105) - rollout_corr/training_log_ppl:np.float64(0.2665938182544778) - rollout_corr/kl:np.float64(0.0010818528244627146) - rollout_corr/k3_kl:np.float64(0.0008180101519030814) - rollout_corr/rollout_ppl:np.float64(1.3230207040905952) - rollout_corr/rollout_log_ppl:np.float64(0.26551196482614614) - rollout_corr/log_ppl_diff:np.float64(0.0010818534283316694) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018223601437057368) - rollout_corr/log_ppl_diff_max:np.float64(0.01393115520477295) - rollout_corr/log_ppl_diff_min:np.float64(-0.010075211524963379) - rollout_corr/ppl_ratio:np.float64(1.0010864462237805) - rollout_corr/chi2_token:np.float64(0.0015508977230638266) - rollout_corr/chi2_seq:np.float64(1.0036336749326438) - actor/pg_loss:np.float64(7.214734796434641e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002760089199682625) - actor/ppo_kl:np.float64(0.0004848889264348344) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21540894359350204) - perf/mfu/actor:np.float64(0.0899665554810477) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27264404296875) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004168026614934206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004168026614934206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:742.140625 - response_length/max:3366.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:742.140625 - response_length_non_aborted/max:3366.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.953603148460388e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6883454918861389) - timing_s/agent_loop/generate_sequences/max:np.float64(21.4869614392519) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.608749296545284) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.4869614392519) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:3366 - timing_s/gen:27.376880433410406 - timing_s/reward:0.07466047257184982 - timing_s/old_log_prob:2.649170648306608 - timing_s/adv:0.12651739083230495 - timing_s/update_actor:10.34231029637158 - timing_s/step:40.673489751294255 - timing_s/stop_profile:0.00011551380157470703 - timing_per_token_ms/update_actor:0.04033977024873851 - timing_per_token_ms/gen:0.1440979453092322 - timing_per_token_ms/adv:0.000493476054420411 - perf/total_num_tokens:256380 - perf/time_per_step:40.673489751294255 - perf/throughput:787.921080683278 (TaskRunner pid=2049544) Training Progress: 81%|████████ | 81/100 [1:03:58<20:28, 64.65s/it] (TaskRunner pid=2049544) step:82 - global_seqlen/min:25371 - global_seqlen/max:44114 - global_seqlen/minmax_diff:18743 - global_seqlen/balanced_min:33520 - global_seqlen/balanced_max:33584 - global_seqlen/mean:33563.875 - actor/entropy:0.2920665442943573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.913862407207489 - training/rollout_probs_diff_mean:0.0031529199331998825 - training/rollout_probs_diff_std:0.009444354102015495 - training/rollout_actor_probs_pearson_corr:0.9993178248405457 - rollout_corr/rollout_is_mean:0.9999819993972778 - rollout_corr/rollout_is_ratio_fraction_high:2.0209470676491037e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.547131175058894e-05 - rollout_corr/rollout_is_max:2.713327169418335 - rollout_corr/rollout_is_min:0.049278318881988525 - rollout_corr/rollout_is_std:0.02968292124569416 - rollout_corr/rollout_is_eff_sample_size:0.9991195807527394 - rollout_corr/rollout_is_seq_mean:0.9999279379844666 - rollout_corr/rollout_is_seq_std:0.0012865856988355517 - rollout_corr/rollout_is_seq_max:1.004623532295227 - rollout_corr/rollout_is_seq_min:0.9942681193351746 - rollout_corr/rollout_is_seq_max_deviation:0.0057318806648254395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3072700020857155) - rollout_corr/training_log_ppl:np.float64(0.24861305610102136) - rollout_corr/kl:np.float64(0.0007281105056762271) - rollout_corr/k3_kl:np.float64(0.0005667303261134293) - rollout_corr/rollout_ppl:np.float64(1.306269062217325) - rollout_corr/rollout_log_ppl:np.float64(0.2478849448962137) - rollout_corr/log_ppl_diff:np.float64(0.0007281112048076466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011902832047780976) - rollout_corr/log_ppl_diff_max:np.float64(0.012343257665634155) - rollout_corr/log_ppl_diff_min:np.float64(-0.00298440083861351) - rollout_corr/ppl_ratio:np.float64(1.0007296884432435) - rollout_corr/chi2_token:np.float64(0.0007541878148913383) - rollout_corr/chi2_seq:np.float64(0.7926566984970123) - actor/pg_loss:np.float64(0.0001337927533313632) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021164456336464355) - actor/ppo_kl:np.float64(0.00017838788005519746) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1409779703244567) - perf/mfu/actor:np.float64(0.09575702121449964) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.184974193573) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00978454202413559 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00978454202413559 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:773.15234375 - response_length/max:2703.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:773.15234375 - response_length_non_aborted/max:2703.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.613321602344513e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2882147133350372) - timing_s/agent_loop/generate_sequences/max:np.float64(18.499180192127824) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.74611435400584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.499180192127824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:240 - timing_s/agent_loop/slowest/response_length:2703 - timing_s/gen:24.018707640469074 - timing_s/reward:0.07196270488202572 - timing_s/old_log_prob:2.514760909602046 - timing_s/adv:0.12197888456285 - timing_s/update_actor:10.373511025682092 - timing_s/step:37.19780015014112 - timing_s/stop_profile:3.86945903301239e-05 - timing_per_token_ms/update_actor:0.03863346762584063 - timing_per_token_ms/gen:0.12135134489215253 - timing_per_token_ms/adv:0.0004542789105952829 - perf/total_num_tokens:268511 - perf/time_per_step:37.19780015014112 - perf/throughput:902.3080629641123 (TaskRunner pid=2049544) Training Progress: 82%|████████▏ | 82/100 [1:04:35<16:55, 56.42s/it] (TaskRunner pid=2049544) step:83 - global_seqlen/min:19276 - global_seqlen/max:41937 - global_seqlen/minmax_diff:22661 - global_seqlen/balanced_min:29583 - global_seqlen/balanced_max:29645 - global_seqlen/mean:29615.125 - actor/entropy:0.34415289759635925 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5622131824493408 - training/rollout_probs_diff_mean:0.003517036559060216 - training/rollout_probs_diff_std:0.009729686193168163 - training/rollout_actor_probs_pearson_corr:0.9993311762809753 - rollout_corr/rollout_is_mean:0.9999920129776001 - rollout_corr/rollout_is_ratio_fraction_high:1.2048701137246098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0121751478873193e-05 - rollout_corr/rollout_is_max:8.23111629486084 - rollout_corr/rollout_is_min:0.2471064180135727 - rollout_corr/rollout_is_std:0.03159145638346672 - rollout_corr/rollout_is_eff_sample_size:0.9990024989924954 - rollout_corr/rollout_is_seq_mean:0.9999005794525146 - rollout_corr/rollout_is_seq_std:0.0016829889500513673 - rollout_corr/rollout_is_seq_max:1.004620909690857 - rollout_corr/rollout_is_seq_min:0.9928184747695923 - rollout_corr/rollout_is_seq_max_deviation:0.007181525230407715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.372836111113429) - rollout_corr/training_log_ppl:np.float64(0.2916473155346466) - rollout_corr/kl:np.float64(0.0007178986236571205) - rollout_corr/k3_kl:np.float64(0.0006351841151968074) - rollout_corr/rollout_ppl:np.float64(1.3718020445667207) - rollout_corr/rollout_log_ppl:np.float64(0.2909294152777875) - rollout_corr/log_ppl_diff:np.float64(0.0007179002568591386) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014387322589755058) - rollout_corr/log_ppl_diff_max:np.float64(0.009183712303638458) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071634650230407715) - rollout_corr/ppl_ratio:np.float64(1.0007200299296528) - rollout_corr/chi2_token:np.float64(0.0011453672777861357) - rollout_corr/chi2_seq:np.float64(0.5700722692999989) - actor/pg_loss:np.float64(0.00010708440095186234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003794109481987107) - actor/ppo_kl:np.float64(5.79457945377726e-05) - actor/pg_clipfrac_lower:np.float64(4.787071247847052e-06) - actor/grad_norm:np.float64(0.27837222814559937) - perf/mfu/actor:np.float64(0.08514566987298042) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15073013305664) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003308121347799897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003308121347799897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.41015625 - response_length/max:2853.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.41015625 - response_length_non_aborted/max:2853.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.170570850372314e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8073275871574879) - timing_s/agent_loop/generate_sequences/max:np.float64(18.502459589391947) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.3145277952207834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.502459589391947) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:2853 - timing_s/gen:24.35667880065739 - timing_s/reward:0.10609085857868195 - timing_s/old_log_prob:2.4029372967779636 - timing_s/adv:0.11332383565604687 - timing_s/update_actor:10.112380102276802 - timing_s/step:37.18459020368755 - timing_s/stop_profile:3.2028183341026306e-05 - timing_per_token_ms/update_actor:0.042682497973066134 - timing_per_token_ms/gen:0.14673316827009206 - timing_per_token_ms/adv:0.0004783190838129455 - perf/total_num_tokens:236921 - perf/time_per_step:37.18459020368755 - perf/throughput:796.4354276267675 (TaskRunner pid=2049544) Training Progress: 83%|████████▎ | 83/100 [1:05:12<14:21, 50.66s/it] (TaskRunner pid=2049544) step:84 - global_seqlen/min:25752 - global_seqlen/max:42945 - global_seqlen/minmax_diff:17193 - global_seqlen/balanced_min:33932 - global_seqlen/balanced_max:34032 - global_seqlen/mean:33969.0 - actor/entropy:0.3499566912651062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138913631439209 - training/rollout_probs_diff_mean:0.0033732259180396795 - training/rollout_probs_diff_std:0.009146650321781635 - training/rollout_actor_probs_pearson_corr:0.9994776844978333 - rollout_corr/rollout_is_mean:0.9999618530273438 - rollout_corr/rollout_is_ratio_fraction_high:4.901384272670839e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4506920453859493e-05 - rollout_corr/rollout_is_max:2.404789447784424 - rollout_corr/rollout_is_min:0.2902381718158722 - rollout_corr/rollout_is_std:0.03017878346145153 - rollout_corr/rollout_is_eff_sample_size:0.9990900697549667 - rollout_corr/rollout_is_seq_mean:1.0001087188720703 - rollout_corr/rollout_is_seq_std:0.001655240310356021 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9946045279502869 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3845429411157966) - rollout_corr/training_log_ppl:np.float64(0.29742183088092133) - rollout_corr/kl:np.float64(0.00039445633141355074) - rollout_corr/k3_kl:np.float64(0.0005463047522766828) - rollout_corr/rollout_ppl:np.float64(1.3839370189234614) - rollout_corr/rollout_log_ppl:np.float64(0.29702737282786984) - rollout_corr/log_ppl_diff:np.float64(0.00039445805305149406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012936670245835558) - rollout_corr/log_ppl_diff_max:np.float64(0.00665166974067688) - rollout_corr/log_ppl_diff_min:np.float64(-0.005310043692588806) - rollout_corr/ppl_ratio:np.float64(1.000396022805944) - rollout_corr/chi2_token:np.float64(0.0013724297750741243) - rollout_corr/chi2_seq:np.float64(1.7039004366379231) - actor/pg_loss:np.float64(0.00012169475667178631) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021371153513882746) - actor/ppo_kl:np.float64(1.5052716766383867e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2066413126885891) - perf/mfu/actor:np.float64(0.09535977782956703) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16190719604492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017204470932483673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017204470932483673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:796.96875 - response_length/max:3594.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:796.96875 - response_length_non_aborted/max:3594.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7237460017204285e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937344156205654) - timing_s/agent_loop/generate_sequences/max:np.float64(23.47761689312756) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.72609610089421) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.47761689312756) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:3594 - timing_s/gen:29.4396906606853 - timing_s/reward:0.07473043724894524 - timing_s/old_log_prob:2.594569843262434 - timing_s/adv:0.12372583709657192 - timing_s/update_actor:10.489614827558398 - timing_s/step:42.81442074663937 - timing_s/stop_profile:2.9224902391433716e-05 - timing_per_token_ms/update_actor:0.038599954471571135 - timing_per_token_ms/gen:0.14429523321121682 - timing_per_token_ms/adv:0.000455289518003812 - perf/total_num_tokens:271752 - perf/time_per_step:42.81442074663937 - perf/throughput:793.4009010892977 (TaskRunner pid=2049544) Training Progress: 84%|████████▍ | 84/100 [1:05:55<12:52, 48.31s/it] (TaskRunner pid=2049544) step:85 - global_seqlen/min:20755 - global_seqlen/max:54433 - global_seqlen/minmax_diff:33678 - global_seqlen/balanced_min:33282 - global_seqlen/balanced_max:33493 - global_seqlen/mean:33418.125 - actor/entropy:0.29106423258781433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967511296272278 - training/rollout_probs_diff_mean:0.003082117298617959 - training/rollout_probs_diff_std:0.009240406565368176 - training/rollout_actor_probs_pearson_corr:0.9993290901184082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:4.992785306967562e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.9971141227870248e-05 - rollout_corr/rollout_is_max:2.3519351482391357 - rollout_corr/rollout_is_min:0.26859748363494873 - rollout_corr/rollout_is_std:0.029412612318992615 - rollout_corr/rollout_is_eff_sample_size:0.9991358839466532 - rollout_corr/rollout_is_seq_mean:0.9999825954437256 - rollout_corr/rollout_is_seq_std:0.0015941840829327703 - rollout_corr/rollout_is_seq_max:1.0068665742874146 - rollout_corr/rollout_is_seq_min:0.9917056560516357 - rollout_corr/rollout_is_seq_max_deviation:0.008294343948364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2718443693593144) - rollout_corr/training_log_ppl:np.float64(0.22443578000093112) - rollout_corr/kl:np.float64(0.0005144908149414817) - rollout_corr/k3_kl:np.float64(0.0004981835849378058) - rollout_corr/rollout_ppl:np.float64(1.2711914060637355) - rollout_corr/rollout_log_ppl:np.float64(0.22392128901265096) - rollout_corr/log_ppl_diff:np.float64(0.0005144909882801585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011518053579493426) - rollout_corr/log_ppl_diff_max:np.float64(0.009792208671569824) - rollout_corr/log_ppl_diff_min:np.float64(-0.005565732717514038) - rollout_corr/ppl_ratio:np.float64(1.000515874940902) - rollout_corr/chi2_token:np.float64(0.000967925414443016) - rollout_corr/chi2_seq:np.float64(1.0396802069153637) - actor/pg_loss:np.float64(-0.0001277067931368947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000293373741897085) - actor/ppo_kl:np.float64(1.70920517623685e-05) - actor/pg_clipfrac_lower:np.float64(5.778476406703703e-06) - actor/grad_norm:np.float64(0.20695016533136368) - perf/mfu/actor:np.float64(0.09336689611238912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0907974243164) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014600777067244053 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014600777067244053 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:782.37890625 - response_length/max:4261.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:782.37890625 - response_length_non_aborted/max:4261.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.2804364562034607e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7515706215053797) - timing_s/agent_loop/generate_sequences/max:np.float64(26.178023107349873) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.071110863718786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.178023107349873) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4261 - timing_s/gen:31.903686491772532 - timing_s/reward:0.07636177726089954 - timing_s/old_log_prob:2.5928792282938957 - timing_s/adv:0.12288173846900463 - timing_s/update_actor:10.525410840287805 - timing_s/step:45.31749305874109 - timing_s/stop_profile:0.00011776015162467957 - timing_per_token_ms/update_actor:0.039370142850204064 - timing_per_token_ms/gen:0.15928826092183063 - timing_per_token_ms/adv:0.0004596373168340707 - perf/total_num_tokens:267345 - perf/time_per_step:45.31749305874109 - perf/throughput:737.4221905144448 (TaskRunner pid=2049544) Training Progress: 85%|████████▌ | 85/100 [1:06:41<11:51, 47.43s/it] (TaskRunner pid=2049544) step:86 - global_seqlen/min:22104 - global_seqlen/max:49662 - global_seqlen/minmax_diff:27558 - global_seqlen/balanced_min:36725 - global_seqlen/balanced_max:36775 - global_seqlen/mean:36762.375 - actor/entropy:0.3210241496562958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24132895469665527 - training/rollout_probs_diff_mean:0.003310859901830554 - training/rollout_probs_diff_std:0.009120251052081585 - training/rollout_actor_probs_pearson_corr:0.9993811845779419 - rollout_corr/rollout_is_mean:0.9998584389686584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2271813577390276e-05 - rollout_corr/rollout_is_max:1.9132159948349 - rollout_corr/rollout_is_min:0.41626274585723877 - rollout_corr/rollout_is_std:0.029449068009853363 - rollout_corr/rollout_is_eff_sample_size:0.9991331468769348 - rollout_corr/rollout_is_seq_mean:0.9998147487640381 - rollout_corr/rollout_is_seq_std:0.0013719926355406642 - rollout_corr/rollout_is_seq_max:1.0041395425796509 - rollout_corr/rollout_is_seq_min:0.9918246865272522 - rollout_corr/rollout_is_seq_max_deviation:0.008175313472747803 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.318041059654206) - rollout_corr/training_log_ppl:np.float64(0.26210865678149275) - rollout_corr/kl:np.float64(0.0005536363921763154) - rollout_corr/k3_kl:np.float64(0.0004948169941201286) - rollout_corr/rollout_ppl:np.float64(1.317284729797393) - rollout_corr/rollout_log_ppl:np.float64(0.2615550198825076) - rollout_corr/log_ppl_diff:np.float64(0.0005536368989851326) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010979093494825065) - rollout_corr/log_ppl_diff_max:np.float64(0.0066015273332595825) - rollout_corr/log_ppl_diff_min:np.float64(-0.002927526831626892) - rollout_corr/ppl_ratio:np.float64(1.0005547057371587) - rollout_corr/chi2_token:np.float64(0.0008581595029681921) - rollout_corr/chi2_seq:np.float64(0.48298407695256174) - actor/pg_loss:np.float64(0.00010274478700011969) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012036708881169034) - actor/ppo_kl:np.float64(-9.579434273687681e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1470967587083578) - perf/mfu/actor:np.float64(0.1030748564778111) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2135009765625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007343573961406946 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007343573961406946 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:876.94921875 - response_length/max:3278.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:876.94921875 - response_length_non_aborted/max:3278.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010039471089839935 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4529647156596184) - timing_s/agent_loop/generate_sequences/max:np.float64(22.549825344234705) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.41551528852142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.549825344234705) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:3278 - timing_s/gen:28.685686687007546 - timing_s/reward:0.07788657583296299 - timing_s/old_log_prob:2.5779322646558285 - timing_s/adv:0.1303798407316208 - timing_s/update_actor:10.655327502638102 - timing_s/step:42.21963391080499 - timing_s/stop_profile:0.00011177733540534973 - timing_per_token_ms/update_actor:0.036230410516996325 - timing_per_token_ms/gen:0.1277764564074118 - timing_per_token_ms/adv:0.0004433195649479284 - perf/total_num_tokens:294099 - perf/time_per_step:42.21963391080499 - perf/throughput:870.7412072228237 (TaskRunner pid=2049544) Training Progress: 86%|████████▌ | 86/100 [1:07:23<10:42, 45.88s/it] (TaskRunner pid=2049544) step:87 - global_seqlen/min:29440 - global_seqlen/max:47768 - global_seqlen/minmax_diff:18328 - global_seqlen/balanced_min:35517 - global_seqlen/balanced_max:36384 - global_seqlen/mean:35692.75 - actor/entropy:0.3466551899909973 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292303800582886 - training/rollout_probs_diff_mean:0.003489995375275612 - training/rollout_probs_diff_std:0.00928904302418232 - training/rollout_actor_probs_pearson_corr:0.9994551539421082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.662482751882635e-06 - rollout_corr/rollout_is_max:1.702067494392395 - rollout_corr/rollout_is_min:0.26140108704566956 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:1.0000789165496826 - rollout_corr/rollout_is_seq_std:0.001175415120087564 - rollout_corr/rollout_is_seq_max:1.0036451816558838 - rollout_corr/rollout_is_seq_min:0.9961031079292297 - rollout_corr/rollout_is_seq_max_deviation:0.0038968920707702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.384202916175127) - rollout_corr/training_log_ppl:np.float64(0.2980515792296501) - rollout_corr/kl:np.float64(0.00042411336519165843) - rollout_corr/k3_kl:np.float64(0.00048466515880818406) - rollout_corr/rollout_ppl:np.float64(1.3836216540075839) - rollout_corr/rollout_log_ppl:np.float64(0.2976274649699917) - rollout_corr/log_ppl_diff:np.float64(0.00042411425965838134) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010522096417844296) - rollout_corr/log_ppl_diff_max:np.float64(0.004691094160079956) - rollout_corr/log_ppl_diff_min:np.float64(-0.005741387605667114) - rollout_corr/ppl_ratio:np.float64(1.0004250849597156) - rollout_corr/chi2_token:np.float64(0.001095710089430213) - rollout_corr/chi2_seq:np.float64(1.8486225868109614) - actor/pg_loss:np.float64(7.82626448199153e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016853076499501185) - actor/ppo_kl:np.float64(3.361720549453773e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18693841621279716) - perf/mfu/actor:np.float64(0.0996124497047341) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18145751953125) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006155060138553381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006155060138553381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:837.8046875 - response_length/max:4550.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:837.8046875 - response_length_non_aborted/max:4550.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001092962920665741 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4342339634895325) - timing_s/agent_loop/generate_sequences/max:np.float64(28.061727035790682) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.031970594347513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.061727035790682) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:4550 - timing_s/gen:33.89907926507294 - timing_s/reward:0.07478075660765171 - timing_s/old_log_prob:2.563651405274868 - timing_s/adv:0.12060143612325191 - timing_s/update_actor:10.524738831445575 - timing_s/step:47.28383187763393 - timing_s/stop_profile:0.00016075000166893005 - timing_per_token_ms/update_actor:0.036858811773558965 - timing_per_token_ms/gen:0.15805387622540745 - timing_per_token_ms/adv:0.00042235970933611136 - perf/total_num_tokens:285542 - perf/time_per_step:47.28383187763393 - perf/throughput:754.8616214601526 (TaskRunner pid=2049544) Training Progress: 87%|████████▋ | 87/100 [1:08:10<10:02, 46.32s/it] (TaskRunner pid=2049544) step:88 - global_seqlen/min:22456 - global_seqlen/max:46812 - global_seqlen/minmax_diff:24356 - global_seqlen/balanced_min:35428 - global_seqlen/balanced_max:35596 - global_seqlen/mean:35553.625 - actor/entropy:0.3236216604709625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511626243591309 - training/rollout_probs_diff_mean:0.0032852201256901026 - training/rollout_probs_diff_std:0.009024620987474918 - training/rollout_actor_probs_pearson_corr:0.999454140663147 - rollout_corr/rollout_is_mean:1.0000885725021362 - rollout_corr/rollout_is_ratio_fraction_high:4.604963123711059e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.604963123711059e-06 - rollout_corr/rollout_is_max:2.0104920864105225 - rollout_corr/rollout_is_min:0.23549965023994446 - rollout_corr/rollout_is_std:0.029910963028669357 - rollout_corr/rollout_is_eff_sample_size:0.999106252989479 - rollout_corr/rollout_is_seq_mean:1.0000383853912354 - rollout_corr/rollout_is_seq_std:0.0014727648813277483 - rollout_corr/rollout_is_seq_max:1.0041821002960205 - rollout_corr/rollout_is_seq_min:0.9893348813056946 - rollout_corr/rollout_is_seq_max_deviation:0.01066511869430542 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338182927109301) - rollout_corr/training_log_ppl:np.float64(0.2705517132417299) - rollout_corr/kl:np.float64(0.0005242268059881816) - rollout_corr/k3_kl:np.float64(0.000508854052498009) - rollout_corr/rollout_ppl:np.float64(1.3374784495681524) - rollout_corr/rollout_log_ppl:np.float64(0.27002748603990767) - rollout_corr/log_ppl_diff:np.float64(0.0005242272018222138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011741442867787555) - rollout_corr/log_ppl_diff_max:np.float64(0.013874843716621399) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045333802700042725) - rollout_corr/ppl_ratio:np.float64(1.0005257281009108) - rollout_corr/chi2_token:np.float64(0.0009958110749721527) - rollout_corr/chi2_seq:np.float64(1.3633345067501068) - actor/pg_loss:np.float64(1.946359407156706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016658261313295952) - actor/ppo_kl:np.float64(7.023494554836418e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13375208340585232) - perf/mfu/actor:np.float64(0.09886611268499279) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15729522705078) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02385486103594303 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02385486103594303 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:848.26953125 - response_length/max:4442.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:848.26953125 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.980984032154083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.990953391417861) - timing_s/agent_loop/generate_sequences/max:np.float64(25.68886986002326) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.6535395380997215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.68886986002326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:4442 - timing_s/gen:31.64418187364936 - timing_s/reward:0.07790577039122581 - timing_s/old_log_prob:2.6047119572758675 - timing_s/adv:0.11337987333536148 - timing_s/update_actor:10.512858772650361 - timing_s/step:45.041199665516615 - timing_s/stop_profile:0.00016839616000652313 - timing_per_token_ms/update_actor:0.036961276004381975 - timing_per_token_ms/gen:0.1457202939516081 - timing_per_token_ms/adv:0.00039862276116486536 - perf/total_num_tokens:284429 - perf/time_per_step:45.041199665516615 - perf/throughput:789.3578604483693 (TaskRunner pid=2049544) Training Progress: 88%|████████▊ | 88/100 [1:08:55<09:11, 45.95s/it] (TaskRunner pid=2049544) step:89 - global_seqlen/min:23526 - global_seqlen/max:43993 - global_seqlen/minmax_diff:20467 - global_seqlen/balanced_min:32615 - global_seqlen/balanced_max:32696 - global_seqlen/mean:32672.25 - actor/entropy:0.3472384810447693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7165358662605286 - training/rollout_probs_diff_mean:0.003400100627914071 - training/rollout_probs_diff_std:0.009368252009153366 - training/rollout_actor_probs_pearson_corr:0.9993821382522583 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:5.2063269322388805e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.123795977444388e-05 - rollout_corr/rollout_is_max:2.0695598125457764 - rollout_corr/rollout_is_min:0.2369716465473175 - rollout_corr/rollout_is_std:0.03049706295132637 - rollout_corr/rollout_is_eff_sample_size:0.9990706743631297 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0016654012724757195 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.9951695799827576 - rollout_corr/rollout_is_seq_max_deviation:0.007972240447998047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3569685937836766) - rollout_corr/training_log_ppl:np.float64(0.27856632304610685) - rollout_corr/kl:np.float64(0.0005558686453759343) - rollout_corr/k3_kl:np.float64(0.0005486493795388014) - rollout_corr/rollout_ppl:np.float64(1.3561613205820322) - rollout_corr/rollout_log_ppl:np.float64(0.27801045119122136) - rollout_corr/log_ppl_diff:np.float64(0.0005558718548854813) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011855626216856763) - rollout_corr/log_ppl_diff_max:np.float64(0.005568467080593109) - rollout_corr/log_ppl_diff_min:np.float64(-0.006579384207725525) - rollout_corr/ppl_ratio:np.float64(1.0005573213566095) - rollout_corr/chi2_token:np.float64(0.0010650090407580137) - rollout_corr/chi2_seq:np.float64(0.8179540059063584) - actor/pg_loss:np.float64(3.63390427082777e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022707523396547913) - actor/ppo_kl:np.float64(6.390776004217003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3162117004394531) - perf/mfu/actor:np.float64(0.09205693492850642) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16839599609375) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01907663233578205 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01907663233578205 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:750.2890625 - response_length/max:3252.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:750.2890625 - response_length_non_aborted/max:3252.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003261677920818329 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3473277054727077) - timing_s/agent_loop/generate_sequences/max:np.float64(20.700606567785144) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.879736434209917) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.700606567785144) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:3252 - timing_s/gen:26.722555236890912 - timing_s/reward:0.0684677753597498 - timing_s/old_log_prob:2.669492883607745 - timing_s/adv:0.1653036493808031 - timing_s/update_actor:10.382515193894506 - timing_s/step:40.15039100125432 - timing_s/stop_profile:0.00012563727796077728 - timing_per_token_ms/update_actor:0.03972222296403869 - timing_per_token_ms/gen:0.13912635357669914 - timing_per_token_ms/adv:0.0006324313805324209 - perf/total_num_tokens:261378 - perf/time_per_step:40.15039100125432 - perf/throughput:813.7467453051529 (TaskRunner pid=2049544) Training Progress: 89%|████████▉ | 89/100 [1:09:36<08:07, 44.35s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2049544) step:90 - global_seqlen/min:22189 - global_seqlen/max:43070 - global_seqlen/minmax_diff:20881 - global_seqlen/balanced_min:29230 - global_seqlen/balanced_max:29298 - global_seqlen/mean:29287.75 - actor/entropy:0.3101608455181122 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3454935550689697 - training/rollout_probs_diff_mean:0.0033272632863372564 - training/rollout_probs_diff_std:0.009299682453274727 - training/rollout_actor_probs_pearson_corr:0.999302327632904 - rollout_corr/rollout_is_mean:0.9999826550483704 - rollout_corr/rollout_is_ratio_fraction_high:1.2050661098328419e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2050661098328419e-05 - rollout_corr/rollout_is_max:2.8188319206237793 - rollout_corr/rollout_is_min:0.3406546115875244 - rollout_corr/rollout_is_std:0.03000447526574135 - rollout_corr/rollout_is_eff_sample_size:0.9991005411997865 - rollout_corr/rollout_is_seq_mean:1.0000338554382324 - rollout_corr/rollout_is_seq_std:0.0013818239094689488 - rollout_corr/rollout_is_seq_max:1.0056010484695435 - rollout_corr/rollout_is_seq_min:0.9940690398216248 - rollout_corr/rollout_is_seq_max_deviation:0.005930960178375244 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3156746202148497) - rollout_corr/training_log_ppl:np.float64(0.2531014566193335) - rollout_corr/kl:np.float64(0.0005285690323812986) - rollout_corr/k3_kl:np.float64(0.0004731722787880699) - rollout_corr/rollout_ppl:np.float64(1.3148961896076798) - rollout_corr/rollout_log_ppl:np.float64(0.25257288754801266) - rollout_corr/log_ppl_diff:np.float64(0.0005285690713208169) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010851562838070095) - rollout_corr/log_ppl_diff_max:np.float64(0.006567090749740601) - rollout_corr/log_ppl_diff_min:np.float64(-0.00448918342590332) - rollout_corr/ppl_ratio:np.float64(1.0005297421012074) - rollout_corr/chi2_token:np.float64(0.0008521552663296461) - rollout_corr/chi2_seq:np.float64(0.22603251412510872) - actor/pg_loss:np.float64(0.00011028861626982689) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012377322673273738) - actor/ppo_kl:np.float64(0.00010799788028137414) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14029540959745646) - perf/mfu/actor:np.float64(0.08421706394502831) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.20226669311523) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6625) - val-aux/sciknoweval/score/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0730352339040406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10800278634919543) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8915625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04701965592013402) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.693225) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9890625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03660810518441162) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9990500000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010677570009542227) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9789875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.048719255966368245) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.989175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036500484252753186) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9618874999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06036258960895739) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9962249999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021651315045931742) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9349000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06689717910953397) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9992000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009789340111118103) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9021374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05987592930423645) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017437187447881502) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01157616637647152 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01157616637647152 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.3046875 - response_length/max:2634.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.3046875 - response_length_non_aborted/max:2634.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011180341243743896 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8763771429657936) - timing_s/agent_loop/generate_sequences/max:np.float64(17.981070213019848) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.232629431753594) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.981070213019848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2634 - timing_s/gen:23.51352482289076 - timing_s/reward:0.0758508276194334 - timing_s/old_log_prob:2.507504915818572 - timing_s/adv:0.1161189116537571 - timing_s/update_actor:10.261307056993246 - timing_s/step:36.5611352995038 - timing_s/testing:112.82421230524778 - timing_s/save_checkpoint:6.677826223894954 - timing_s/stop_profile:0.0001322384923696518 - timing_per_token_ms/update_actor:0.04379521752692357 - timing_per_token_ms/gen:0.14167675802809465 - timing_per_token_ms/adv:0.0004955950510612675 - perf/total_num_tokens:234302 - perf/time_per_step:36.5611352995038 - perf/throughput:801.0623783993241 (TaskRunner pid=2049544) Training Progress: 90%|█████████ | 90/100 [1:12:12<12:58, 77.88s/it] (TaskRunner pid=2049544) step:91 - global_seqlen/min:21497 - global_seqlen/max:45454 - global_seqlen/minmax_diff:23957 - global_seqlen/balanced_min:32904 - global_seqlen/balanced_max:33139 - global_seqlen/mean:33002.875 - actor/entropy:0.26041534543037415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34585317969322205 - training/rollout_probs_diff_mean:0.002946822438389063 - training/rollout_probs_diff_std:0.008950787596404552 - training/rollout_actor_probs_pearson_corr:0.9992419481277466 - rollout_corr/rollout_is_mean:0.9999063611030579 - rollout_corr/rollout_is_ratio_fraction_high:5.014869202568661e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5044607607705984e-05 - rollout_corr/rollout_is_max:5.096095561981201 - rollout_corr/rollout_is_min:0.37255313992500305 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.001477584009990096 - rollout_corr/rollout_is_seq_max:1.0067633390426636 - rollout_corr/rollout_is_seq_min:0.995822548866272 - rollout_corr/rollout_is_seq_max_deviation:0.006763339042663574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2352091469801962) - rollout_corr/training_log_ppl:np.float64(0.20221140759531409) - rollout_corr/kl:np.float64(0.00045496249386900445) - rollout_corr/k3_kl:np.float64(0.00047493476955651204) - rollout_corr/rollout_ppl:np.float64(1.2346553695388138) - rollout_corr/rollout_log_ppl:np.float64(0.2017564445413882) - rollout_corr/log_ppl_diff:np.float64(0.0004549630539258942) - rollout_corr/log_ppl_abs_diff:np.float64(0.001167977330624126) - rollout_corr/log_ppl_diff_max:np.float64(0.009916573762893677) - rollout_corr/log_ppl_diff_min:np.float64(-0.006213173270225525) - rollout_corr/ppl_ratio:np.float64(1.0004563292022794) - rollout_corr/chi2_token:np.float64(0.0009643365629017353) - rollout_corr/chi2_seq:np.float64(0.6764775053597987) - actor/pg_loss:np.float64(5.630298983305693e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001349058953792337) - actor/ppo_kl:np.float64(3.4980036296872186e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12929855473339558) - perf/mfu/actor:np.float64(0.09015558565254489) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.73992156982422) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012762214988470078 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012762214988470078 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:778.93359375 - response_length/max:5011.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:778.93359375 - response_length_non_aborted/max:5011.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5048614740371704e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9828272927552462) - timing_s/agent_loop/generate_sequences/max:np.float64(30.033628037199378) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.231723692668311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.033628037199378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:5011 - timing_s/gen:35.68446674197912 - timing_s/reward:0.0753241665661335 - timing_s/old_log_prob:2.7556536067277193 - timing_s/adv:0.12121403589844704 - timing_s/update_actor:10.860587490722537 - timing_s/step:49.55868074670434 - timing_s/stop_profile:0.00011924095451831818 - timing_per_token_ms/update_actor:0.04113500524849175 - timing_per_token_ms/gen:0.17895292914480998 - timing_per_token_ms/adv:0.0004591040776691691 - perf/total_num_tokens:264023 - perf/time_per_step:49.55868074670434 - perf/throughput:665.9353013991337 (TaskRunner pid=2049544) Training Progress: 91%|█████████ | 91/100 [1:13:02<10:24, 69.39s/it] (TaskRunner pid=2049544) step:92 - global_seqlen/min:27995 - global_seqlen/max:54733 - global_seqlen/minmax_diff:26738 - global_seqlen/balanced_min:38799 - global_seqlen/balanced_max:39948 - global_seqlen/mean:39032.875 - actor/entropy:0.365276038646698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4553599953651428 - training/rollout_probs_diff_mean:0.0034760674461722374 - training/rollout_probs_diff_std:0.009343450888991356 - training/rollout_actor_probs_pearson_corr:0.999370813369751 - rollout_corr/rollout_is_mean:0.9999127984046936 - rollout_corr/rollout_is_ratio_fraction_high:8.297963177028578e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.0744908397318795e-05 - rollout_corr/rollout_is_max:3.3402388095855713 - rollout_corr/rollout_is_min:0.3921363651752472 - rollout_corr/rollout_is_std:0.03065885230898857 - rollout_corr/rollout_is_eff_sample_size:0.999060798471362 - rollout_corr/rollout_is_seq_mean:0.9999949932098389 - rollout_corr/rollout_is_seq_std:0.001438560662791133 - rollout_corr/rollout_is_seq_max:1.005057454109192 - rollout_corr/rollout_is_seq_min:0.9950283765792847 - rollout_corr/rollout_is_seq_max_deviation:0.0050574541091918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4253517426550388) - rollout_corr/training_log_ppl:np.float64(0.3268485037915525) - rollout_corr/kl:np.float64(0.0007143817989492263) - rollout_corr/k3_kl:np.float64(0.0007667658379375553) - rollout_corr/rollout_ppl:np.float64(1.4242633627727628) - rollout_corr/rollout_log_ppl:np.float64(0.3261341213074047) - rollout_corr/log_ppl_diff:np.float64(0.0007143824841477908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013714263404835947) - rollout_corr/log_ppl_diff_max:np.float64(0.013571739196777344) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037727057933807373) - rollout_corr/ppl_ratio:np.float64(1.000716715119779) - rollout_corr/chi2_token:np.float64(0.001422470435500145) - rollout_corr/chi2_seq:np.float64(0.832664096262306) - actor/pg_loss:np.float64(2.3467233404517174e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434544945548623) - actor/ppo_kl:np.float64(0.00016380211024369373) - actor/pg_clipfrac_lower:np.float64(2.0755844616360264e-06) - actor/grad_norm:np.float64(0.215729471296072) - perf/mfu/actor:np.float64(0.10647986423451218) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.4896354675293) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011413848027586937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011413848027586937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:941.49609375 - response_length/max:5473.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:941.49609375 - response_length_non_aborted/max:5473.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.439482629299164e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7557445913553238) - timing_s/agent_loop/generate_sequences/max:np.float64(32.87127552181482) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.810325469457894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.87127552181482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:5473 - timing_s/gen:38.770385617390275 - timing_s/reward:0.08092910423874855 - timing_s/old_log_prob:2.7198645491153 - timing_s/adv:0.12597152031958103 - timing_s/update_actor:10.861796030774713 - timing_s/step:52.657433899119496 - timing_s/stop_profile:0.00011220574378967285 - timing_per_token_ms/update_actor:0.034784127580836384 - timing_per_token_ms/gen:0.1608576178098782 - timing_per_token_ms/adv:0.00040341481481821745 - perf/total_num_tokens:312263 - perf/time_per_step:52.657433899119496 - perf/throughput:741.2604851724968 (TaskRunner pid=2049544) Training Progress: 92%|█████████▏| 92/100 [1:13:54<08:35, 64.39s/it] (TaskRunner pid=2049544) step:93 - global_seqlen/min:17843 - global_seqlen/max:51915 - global_seqlen/minmax_diff:34072 - global_seqlen/balanced_min:30929 - global_seqlen/balanced_max:31113 - global_seqlen/mean:31065.0 - actor/entropy:0.4133561849594116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311214327812195 - training/rollout_probs_diff_mean:0.0036772009916603565 - training/rollout_probs_diff_std:0.009407786652445793 - training/rollout_actor_probs_pearson_corr:0.9994404911994934 - rollout_corr/rollout_is_mean:0.9999762177467346 - rollout_corr/rollout_is_ratio_fraction_high:5.474292720464291e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6422878616140224e-05 - rollout_corr/rollout_is_max:2.111367702484131 - rollout_corr/rollout_is_min:0.21797895431518555 - rollout_corr/rollout_is_std:0.031457215547561646 - rollout_corr/rollout_is_eff_sample_size:0.9990114219410976 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0018004179000854492 - rollout_corr/rollout_is_seq_max:1.007155179977417 - rollout_corr/rollout_is_seq_min:0.9935778975486755 - rollout_corr/rollout_is_seq_max_deviation:0.007155179977416992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4254543129354715) - rollout_corr/training_log_ppl:np.float64(0.3189181691413978) - rollout_corr/kl:np.float64(0.0006560000657124254) - rollout_corr/k3_kl:np.float64(0.0006174342987037562) - rollout_corr/rollout_ppl:np.float64(1.4244641666300595) - rollout_corr/rollout_log_ppl:np.float64(0.31826216776971705) - rollout_corr/log_ppl_diff:np.float64(0.0006560013716807589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014309252292150632) - rollout_corr/log_ppl_diff_max:np.float64(0.00973115861415863) - rollout_corr/log_ppl_diff_min:np.float64(-0.004590287804603577) - rollout_corr/ppl_ratio:np.float64(1.0006579444743693) - rollout_corr/chi2_token:np.float64(0.0011531137861311436) - rollout_corr/chi2_seq:np.float64(0.8776039613876492) - actor/pg_loss:np.float64(0.00012877536937594414) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000180150422920633) - actor/ppo_kl:np.float64(9.64217530006195e-05) - actor/pg_clipfrac_lower:np.float64(3.570612534531392e-06) - actor/grad_norm:np.float64(0.18000993877649307) - perf/mfu/actor:np.float64(0.08642174822282235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.50779724121094) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016780760139226913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016780760139226913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:713.5625 - response_length/max:4647.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:713.5625 - response_length_non_aborted/max:4647.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010390952229499817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7623797114938498) - timing_s/agent_loop/generate_sequences/max:np.float64(27.237358892336488) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.646888445196964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.237358892336488) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4647 - timing_s/gen:33.228857627138495 - timing_s/reward:0.07251898013055325 - timing_s/old_log_prob:2.6381760984659195 - timing_s/adv:0.12340777926146984 - timing_s/update_actor:10.632954278960824 - timing_s/step:46.78136018849909 - timing_s/stop_profile:0.0001156739890575409 - timing_per_token_ms/update_actor:0.04278510493707075 - timing_per_token_ms/gen:0.18190449344802978 - timing_per_token_ms/adv:0.0004965708162782466 - perf/total_num_tokens:248520 - perf/time_per_step:46.78136018849909 - perf/throughput:664.0465320979944 (TaskRunner pid=2049544) Training Progress: 93%|█████████▎| 93/100 [1:14:41<06:53, 59.12s/it] (TaskRunner pid=2049544) step:94 - global_seqlen/min:22853 - global_seqlen/max:49175 - global_seqlen/minmax_diff:26322 - global_seqlen/balanced_min:33215 - global_seqlen/balanced_max:33359 - global_seqlen/mean:33330.0 - actor/entropy:0.36058807373046875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8867406845092773 - training/rollout_probs_diff_mean:0.003440327476710081 - training/rollout_probs_diff_std:0.009632454253733158 - training/rollout_actor_probs_pearson_corr:0.9993698000907898 - rollout_corr/rollout_is_mean:0.9999211430549622 - rollout_corr/rollout_is_ratio_fraction_high:1.0051463505078573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.523158713709563e-05 - rollout_corr/rollout_is_max:2.117000102996826 - rollout_corr/rollout_is_min:0.0013813151745125651 - rollout_corr/rollout_is_std:0.030682172626256943 - rollout_corr/rollout_is_eff_sample_size:0.9990594896328889 - rollout_corr/rollout_is_seq_mean:0.9999591112136841 - rollout_corr/rollout_is_seq_std:0.0020710721146315336 - rollout_corr/rollout_is_seq_max:1.011606216430664 - rollout_corr/rollout_is_seq_min:0.9853842854499817 - rollout_corr/rollout_is_seq_max_deviation:0.01461571455001831 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3442378360778093) - rollout_corr/training_log_ppl:np.float64(0.2689030548208393) - rollout_corr/kl:np.float64(0.0007270200237998736) - rollout_corr/k3_kl:np.float64(0.0005578451756775848) - rollout_corr/rollout_ppl:np.float64(1.3432344142347574) - rollout_corr/rollout_log_ppl:np.float64(0.26817603285599034) - rollout_corr/log_ppl_diff:np.float64(0.0007270219648489729) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013523521920433268) - rollout_corr/log_ppl_diff_max:np.float64(0.018366605043411255) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493505597114563) - rollout_corr/ppl_ratio:np.float64(1.0007292504888028) - rollout_corr/chi2_token:np.float64(0.0007120152004063129) - rollout_corr/chi2_seq:np.float64(0.3402676989790052) - actor/pg_loss:np.float64(-4.822376649826765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.676928118551587e-05) - actor/ppo_kl:np.float64(0.00014517965372551345) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16034816205501556) - perf/mfu/actor:np.float64(0.09308032359557787) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.54820251464844) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0465502105653286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0465502105653286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:777.25 - response_length/max:3614.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:777.25 - response_length_non_aborted/max:3614.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001496635377407074 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8505959454923868) - timing_s/agent_loop/generate_sequences/max:np.float64(23.250541422516108) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.11456365948834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.250541422516108) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:293 - timing_s/agent_loop/slowest/response_length:3614 - timing_s/gen:29.206484995782375 - timing_s/reward:0.07558401487767696 - timing_s/old_log_prob:2.6541784573346376 - timing_s/adv:0.12619186379015446 - timing_s/update_actor:10.563141915947199 - timing_s/step:42.71316540054977 - timing_s/stop_profile:0.00011851266026496887 - timing_per_token_ms/update_actor:0.03961574375917792 - timing_per_token_ms/gen:0.14678395884821474 - timing_per_token_ms/adv:0.0004732668158946687 - perf/total_num_tokens:266640 - perf/time_per_step:42.71316540054977 - perf/throughput:780.3214696790185 (TaskRunner pid=2049544) Training Progress: 94%|█████████▍| 94/100 [1:15:24<05:25, 54.21s/it] (TaskRunner pid=2049544) step:95 - global_seqlen/min:27066 - global_seqlen/max:58256 - global_seqlen/minmax_diff:31190 - global_seqlen/balanced_min:37178 - global_seqlen/balanced_max:37249 - global_seqlen/mean:37221.375 - actor/entropy:0.3698161244392395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553989887237549 - training/rollout_probs_diff_mean:0.0034687381703406572 - training/rollout_probs_diff_std:0.009108307771384716 - training/rollout_actor_probs_pearson_corr:0.999431848526001 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3214753380452748e-05 - rollout_corr/rollout_is_max:1.8554537296295166 - rollout_corr/rollout_is_min:0.4261438846588135 - rollout_corr/rollout_is_std:0.030212340876460075 - rollout_corr/rollout_is_eff_sample_size:0.9990878089036473 - rollout_corr/rollout_is_seq_mean:0.9999582767486572 - rollout_corr/rollout_is_seq_std:0.0013638396048918366 - rollout_corr/rollout_is_seq_max:1.0052945613861084 - rollout_corr/rollout_is_seq_min:0.9958161115646362 - rollout_corr/rollout_is_seq_max_deviation:0.0052945613861083984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3786341231316328) - rollout_corr/training_log_ppl:np.float64(0.30179976880026516) - rollout_corr/kl:np.float64(0.0004993855567150263) - rollout_corr/k3_kl:np.float64(0.0005308137937447555) - rollout_corr/rollout_ppl:np.float64(1.377938257995993) - rollout_corr/rollout_log_ppl:np.float64(0.30130038285278715) - rollout_corr/log_ppl_diff:np.float64(0.0004993859474780038) - rollout_corr/log_ppl_abs_diff:np.float64(0.001156414728029631) - rollout_corr/log_ppl_diff_max:np.float64(0.0048898980021476746) - rollout_corr/log_ppl_diff_min:np.float64(-0.00559312105178833) - rollout_corr/ppl_ratio:np.float64(1.0005006100982428) - rollout_corr/chi2_token:np.float64(0.001144631765782833) - rollout_corr/chi2_seq:np.float64(0.9150484425481409) - actor/pg_loss:np.float64(-2.8662499971687794e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000206823247367538) - actor/ppo_kl:np.float64(-2.0183409755958337e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1875382363796234) - perf/mfu/actor:np.float64(0.10496417728509784) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.40451049804688) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.021443139761686325 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.021443139761686325 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:886.79296875 - response_length/max:3209.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:886.79296875 - response_length_non_aborted/max:3209.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.542517364025116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.471508091315627) - timing_s/agent_loop/generate_sequences/max:np.float64(22.023407490924) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.383664465822221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.023407490924) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:3175 - timing_s/gen:28.187608854845166 - timing_s/reward:0.08735943771898746 - timing_s/old_log_prob:2.6067604515701532 - timing_s/adv:0.13234028778970242 - timing_s/update_actor:10.463114799931645 - timing_s/step:41.565891321748495 - timing_s/stop_profile:0.00011608004570007324 - timing_per_token_ms/update_actor:0.035138125606360746 - timing_per_token_ms/gen:0.12416409575782276 - timing_per_token_ms/adv:0.00044443645549668175 - perf/total_num_tokens:297771 - perf/time_per_step:41.565891321748495 - perf/throughput:895.4788124686426 (TaskRunner pid=2049544) Training Progress: 95%|█████████▌| 95/100 [1:16:06<04:12, 50.43s/it] (TaskRunner pid=2049544) step:96 - global_seqlen/min:20271 - global_seqlen/max:71208 - global_seqlen/minmax_diff:50937 - global_seqlen/balanced_min:37576 - global_seqlen/balanced_max:37647 - global_seqlen/mean:37613.5 - actor/entropy:0.334806889295578 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31711164116859436 - training/rollout_probs_diff_mean:0.003466662485152483 - training/rollout_probs_diff_std:0.009574675001204014 - training/rollout_actor_probs_pearson_corr:0.9992694854736328 - rollout_corr/rollout_is_mean:1.0000592470169067 - rollout_corr/rollout_is_ratio_fraction_high:8.720068763068411e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.720068763068411e-06 - rollout_corr/rollout_is_max:2.086963415145874 - rollout_corr/rollout_is_min:0.4454006552696228 - rollout_corr/rollout_is_std:0.030977971851825714 - rollout_corr/rollout_is_eff_sample_size:0.9990414042159874 - rollout_corr/rollout_is_seq_mean:1.0001949071884155 - rollout_corr/rollout_is_seq_std:0.0014157574623823166 - rollout_corr/rollout_is_seq_max:1.0063108205795288 - rollout_corr/rollout_is_seq_min:0.9960592985153198 - rollout_corr/rollout_is_seq_max_deviation:0.006310820579528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3516691238619387) - rollout_corr/training_log_ppl:np.float64(0.27739123623177875) - rollout_corr/kl:np.float64(0.00038778805755357304) - rollout_corr/k3_kl:np.float64(0.0005165411663199393) - rollout_corr/rollout_ppl:np.float64(1.3510792162269354) - rollout_corr/rollout_log_ppl:np.float64(0.277003448762116) - rollout_corr/log_ppl_diff:np.float64(0.0003877874696627259) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011570610222406685) - rollout_corr/log_ppl_diff_max:np.float64(0.00403740257024765) - rollout_corr/log_ppl_diff_min:np.float64(-0.005538493394851685) - rollout_corr/ppl_ratio:np.float64(1.000388921238482) - rollout_corr/chi2_token:np.float64(0.0012935972772538662) - rollout_corr/chi2_seq:np.float64(2.2381823365576565) - actor/pg_loss:np.float64(5.385885015130043e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015849828230329877) - actor/ppo_kl:np.float64(8.225317838395085e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11362059134989977) - perf/mfu/actor:np.float64(0.1031450502424441) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.59148025512695) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007041455246508121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007041455246508121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:895.921875 - response_length/max:4074.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:895.921875 - response_length_non_aborted/max:4074.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.657893002033234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2360860873013735) - timing_s/agent_loop/generate_sequences/max:np.float64(26.039900217205286) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.459900658301194) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.039900217205286) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3982 - timing_s/gen:32.00374800339341 - timing_s/reward:0.0823327787220478 - timing_s/old_log_prob:2.7716220505535603 - timing_s/adv:0.1664755642414093 - timing_s/update_actor:10.801547262817621 - timing_s/step:45.91564348526299 - timing_s/stop_profile:0.00011668726801872253 - timing_per_token_ms/update_actor:0.035896510770127815 - timing_per_token_ms/gen:0.13953743526828777 - timing_per_token_ms/adv:0.0005532440621100446 - perf/total_num_tokens:300908 - perf/time_per_step:45.91564348526299 - perf/throughput:819.1870383363432 (TaskRunner pid=2049544) Training Progress: 96%|█████████▌| 96/100 [1:16:52<03:16, 49.09s/it] (TaskRunner pid=2049544) step:97 - global_seqlen/min:20006 - global_seqlen/max:63623 - global_seqlen/minmax_diff:43617 - global_seqlen/balanced_min:40703 - global_seqlen/balanced_max:40790 - global_seqlen/mean:40767.5 - actor/entropy:0.4114430248737335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3968365788459778 - training/rollout_probs_diff_mean:0.0036807965952903032 - training/rollout_probs_diff_std:0.009112636558711529 - training/rollout_actor_probs_pearson_corr:0.9994710087776184 - rollout_corr/rollout_is_mean:0.9999697208404541 - rollout_corr/rollout_is_ratio_fraction_high:3.885426394845126e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5541705579380505e-05 - rollout_corr/rollout_is_max:6.432518005371094 - rollout_corr/rollout_is_min:0.14259245991706848 - rollout_corr/rollout_is_std:0.030860379338264465 - rollout_corr/rollout_is_eff_sample_size:0.9990483051530745 - rollout_corr/rollout_is_seq_mean:1.000016450881958 - rollout_corr/rollout_is_seq_std:0.0015233448939397931 - rollout_corr/rollout_is_seq_max:1.0063254833221436 - rollout_corr/rollout_is_seq_min:0.994326114654541 - rollout_corr/rollout_is_seq_max_deviation:0.006325483322143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3867830368690193) - rollout_corr/training_log_ppl:np.float64(0.30247966502793133) - rollout_corr/kl:np.float64(0.0005262127845906406) - rollout_corr/k3_kl:np.float64(0.0005436497510515892) - rollout_corr/rollout_ppl:np.float64(1.386001723818481) - rollout_corr/rollout_log_ppl:np.float64(0.30195345119864214) - rollout_corr/log_ppl_diff:np.float64(0.0005262138292891905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011905098654096946) - rollout_corr/log_ppl_diff_max:np.float64(0.007206156849861145) - rollout_corr/log_ppl_diff_min:np.float64(-0.003447234630584717) - rollout_corr/ppl_ratio:np.float64(1.0005276133306324) - rollout_corr/chi2_token:np.float64(0.001151977339759469) - rollout_corr/chi2_seq:np.float64(0.950474871089682) - actor/pg_loss:np.float64(6.92273024469614e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019685469499108876) - actor/ppo_kl:np.float64(3.492400511362348e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16588949039578438) - perf/mfu/actor:np.float64(0.11018383492360453) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.53912734985352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.043718818575143814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.043718818575143814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1005.359375 - response_length/max:4322.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1005.359375 - response_length_non_aborted/max:4322.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012983568012714386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.021046968176961) - timing_s/agent_loop/generate_sequences/max:np.float64(28.757525622844696) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.163475782763271) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.757525622844696) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:4322 - timing_s/gen:34.78881582431495 - timing_s/reward:0.08717712201178074 - timing_s/old_log_prob:2.7238584086298943 - timing_s/adv:0.1590470802038908 - timing_s/update_actor:10.954232659190893 - timing_s/step:48.80308444797993 - timing_s/stop_profile:0.00013154186308383942 - timing_per_token_ms/update_actor:0.0335875165854875 - timing_per_token_ms/gen:0.13516938837291917 - timing_per_token_ms/adv:0.000487665052443401 - perf/total_num_tokens:326140 - perf/time_per_step:48.80308444797993 - perf/throughput:835.3467913171513 (TaskRunner pid=2049544) Training Progress: 97%|█████████▋| 97/100 [1:17:40<02:27, 49.03s/it] (TaskRunner pid=2049544) step:98 - global_seqlen/min:23082 - global_seqlen/max:43864 - global_seqlen/minmax_diff:20782 - global_seqlen/balanced_min:32541 - global_seqlen/balanced_max:33196 - global_seqlen/mean:32691.375 - actor/entropy:0.3132493495941162 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6835976243019104 - training/rollout_probs_diff_mean:0.0034196285996586084 - training/rollout_probs_diff_std:0.009468399919569492 - training/rollout_actor_probs_pearson_corr:0.9993211627006531 - rollout_corr/rollout_is_mean:1.0001163482666016 - rollout_corr/rollout_is_ratio_fraction_high:1.5585144865326583e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5585144865326583e-05 - rollout_corr/rollout_is_max:5.03674840927124 - rollout_corr/rollout_is_min:0.0648747980594635 - rollout_corr/rollout_is_std:0.03038937970995903 - rollout_corr/rollout_is_eff_sample_size:0.9990775757045947 - rollout_corr/rollout_is_seq_mean:1.0001397132873535 - rollout_corr/rollout_is_seq_std:0.001354765729047358 - rollout_corr/rollout_is_seq_max:1.0069159269332886 - rollout_corr/rollout_is_seq_min:0.9959137439727783 - rollout_corr/rollout_is_seq_max_deviation:0.006915926933288574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2990983063355088) - rollout_corr/training_log_ppl:np.float64(0.24720221558527555) - rollout_corr/kl:np.float64(0.0005138034307572426) - rollout_corr/k3_kl:np.float64(0.0005089092589116717) - rollout_corr/rollout_ppl:np.float64(1.298400120344013) - rollout_corr/rollout_log_ppl:np.float64(0.24668841203674674) - rollout_corr/log_ppl_diff:np.float64(0.0005138035485288128) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011324260121909901) - rollout_corr/log_ppl_diff_max:np.float64(0.00813785195350647) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736410617828369) - rollout_corr/ppl_ratio:np.float64(1.0005151100922376) - rollout_corr/chi2_token:np.float64(0.0009681761730462313) - rollout_corr/chi2_seq:np.float64(4.5980356687214226) - actor/pg_loss:np.float64(-9.167857933789492e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002514609163881687) - actor/ppo_kl:np.float64(0.00017433107315412144) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1871425360441208) - perf/mfu/actor:np.float64(0.09212177950831188) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56390380859375) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003707965661305934 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0003707965661305934 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:751.91796875 - response_length/max:4148.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.91796875 - response_length_non_aborted/max:4148.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010955892503261566 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1268691644072533) - timing_s/agent_loop/generate_sequences/max:np.float64(24.762351796031) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.324899325562001) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.762351796031) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:4148 - timing_s/gen:30.538867769762874 - timing_s/reward:0.07450433447957039 - timing_s/old_log_prob:2.632541459053755 - timing_s/adv:0.1884304340928793 - timing_s/update_actor:10.359910435974598 - timing_s/step:43.87974337860942 - timing_s/stop_profile:0.000130457803606987 - timing_per_token_ms/update_actor:0.03961255237801484 - timing_per_token_ms/gen:0.15865088637787156 - timing_per_token_ms/adv:0.0007204898619776597 - perf/total_num_tokens:261531 - perf/time_per_step:43.87974337860942 - perf/throughput:745.0220188830104 (TaskRunner pid=2049544) Training Progress: 98%|█████████▊| 98/100 [1:18:24<01:34, 47.50s/it] (TaskRunner pid=2049544) step:99 - global_seqlen/min:21119 - global_seqlen/max:73608 - global_seqlen/minmax_diff:52489 - global_seqlen/balanced_min:37066 - global_seqlen/balanced_max:37818 - global_seqlen/mean:37233.375 - actor/entropy:0.36280471086502075 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29852432012557983 - training/rollout_probs_diff_mean:0.0034003094770014286 - training/rollout_probs_diff_std:0.009052536450326443 - training/rollout_actor_probs_pearson_corr:0.9994275569915771 - rollout_corr/rollout_is_mean:0.9999655485153198 - rollout_corr/rollout_is_ratio_fraction_high:4.338112375990022e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7352449503960088e-05 - rollout_corr/rollout_is_max:2.0474398136138916 - rollout_corr/rollout_is_min:0.40364018082618713 - rollout_corr/rollout_is_std:0.029825152829289436 - rollout_corr/rollout_is_eff_sample_size:0.9991112508590329 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.0013986814301460981 - rollout_corr/rollout_is_seq_max:1.0060322284698486 - rollout_corr/rollout_is_seq_min:0.9942169785499573 - rollout_corr/rollout_is_seq_max_deviation:0.006032228469848633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2896207650192082) - rollout_corr/training_log_ppl:np.float64(0.2305254971361137) - rollout_corr/kl:np.float64(0.0005066159342224097) - rollout_corr/k3_kl:np.float64(0.0004654821964606981) - rollout_corr/rollout_ppl:np.float64(1.2889360883273184) - rollout_corr/rollout_log_ppl:np.float64(0.23001887879217975) - rollout_corr/log_ppl_diff:np.float64(0.0005066183439339511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010047785108326934) - rollout_corr/log_ppl_diff_max:np.float64(0.0074303895235061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489123821258545) - rollout_corr/ppl_ratio:np.float64(1.0005076131783426) - rollout_corr/chi2_token:np.float64(0.000825623283162713) - rollout_corr/chi2_seq:np.float64(0.5253964632283896) - actor/pg_loss:np.float64(7.684435695409775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010054797257907921) - actor/ppo_kl:np.float64(4.189955548028479e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08311776397749782) - perf/mfu/actor:np.float64(0.10069730946959708) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.47393417358398) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.86328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.86328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017708174884319305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017708174884319305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:900.44921875 - response_length/max:6057.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:900.44921875 - response_length_non_aborted/max:6057.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011433474719524384 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8707099817693233) - timing_s/agent_loop/generate_sequences/max:np.float64(35.631088424474) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.754026362330478) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.631088424474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:6057 - timing_s/gen:41.34765339829028 - timing_s/reward:0.08056613802909851 - timing_s/old_log_prob:2.6803389824926853 - timing_s/adv:0.13033771142363548 - timing_s/update_actor:10.983067074790597 - timing_s/step:55.307873111218214 - timing_s/stop_profile:0.00011896714568138123 - timing_per_token_ms/update_actor:0.03687238624886475 - timing_per_token_ms/gen:0.17937077152588887 - timing_per_token_ms/adv:0.000437570161930108 - perf/total_num_tokens:297867 - perf/time_per_step:55.307873111218214 - perf/throughput:673.2020760430919 (TaskRunner pid=2049544) Training Progress: 99%|█████████▉| 99/100 [1:19:20<00:49, 49.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We use the formula for the electric field due to a point charge: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Solving for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2049544) step:100 - global_seqlen/min:17010 - global_seqlen/max:43433 - global_seqlen/minmax_diff:26423 - global_seqlen/balanced_min:31299 - global_seqlen/balanced_max:31491 - global_seqlen/mean:31419.0 - actor/entropy:0.37953582406044006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.630595862865448 - training/rollout_probs_diff_mean:0.0036211644764989614 - training/rollout_probs_diff_std:0.00978316180408001 - training/rollout_actor_probs_pearson_corr:0.999374508857727 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:1.0963950444420334e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.644592521188315e-05 - rollout_corr/rollout_is_max:2.3022942543029785 - rollout_corr/rollout_is_min:0.3131349980831146 - rollout_corr/rollout_is_std:0.031683772802352905 - rollout_corr/rollout_is_eff_sample_size:0.998997264270168 - rollout_corr/rollout_is_seq_mean:1.0000271797180176 - rollout_corr/rollout_is_seq_std:0.0016033316496759653 - rollout_corr/rollout_is_seq_max:1.0072500705718994 - rollout_corr/rollout_is_seq_min:0.9959959983825684 - rollout_corr/rollout_is_seq_max_deviation:0.007250070571899414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3458213401027024) - rollout_corr/training_log_ppl:np.float64(0.26966726542741526) - rollout_corr/kl:np.float64(0.00037315004849824973) - rollout_corr/k3_kl:np.float64(0.0005444729254122649) - rollout_corr/rollout_ppl:np.float64(1.3452776027843356) - rollout_corr/rollout_log_ppl:np.float64(0.2692941162531497) - rollout_corr/log_ppl_diff:np.float64(0.0003731491742655635) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012809229083359241) - rollout_corr/log_ppl_diff_max:np.float64(0.0049625784158706665) - rollout_corr/log_ppl_diff_min:np.float64(-0.0106421560049057) - rollout_corr/ppl_ratio:np.float64(1.0003747479058802) - rollout_corr/chi2_token:np.float64(0.0014326744712889194) - rollout_corr/chi2_seq:np.float64(1.9170411999803036) - actor/pg_loss:np.float64(-7.347844075411558e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001629320678375734) - actor/ppo_kl:np.float64(-0.00012778303023264215) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12031856318935752) - perf/mfu/actor:np.float64(0.0881824181802986) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.57986450195312) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/score/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05584489583302406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08949529688241312) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8910875) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.054003469751280585) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7157250000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9859375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03383512807052093) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99745) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.014287439995541685) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9741250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.042452979144002284) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9857875) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03387729307844336) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998374999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.002749370152057837) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9560875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04764508939236297) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9946999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021483511676811038) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.933025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0436722700672832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9988624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009237722502874987) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9139250000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.029568343249240757) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017543234207183663) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018172748386859894 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018172748386859894 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:712.5625 - response_length/max:3652.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:712.5625 - response_length_non_aborted/max:3652.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.623674511909485e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2800081353634596) - timing_s/agent_loop/generate_sequences/max:np.float64(22.433935675770044) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.539074072250514) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.433935675770044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3652 - timing_s/gen:28.36895477026701 - timing_s/reward:0.07503514550626278 - timing_s/old_log_prob:2.573247427120805 - timing_s/adv:0.12070851400494576 - timing_s/update_actor:10.477075356990099 - timing_s/step:41.70409296452999 - timing_s/testing:119.00530271604657 - timing_s/save_checkpoint:6.902973407879472 - timing_s/stop_profile:0.00019830837845802307 - timing_per_token_ms/update_actor:0.041682880410699334 - timing_per_token_ms/gen:0.15551790835380125 - timing_per_token_ms/adv:0.00048023693467704956 - perf/total_num_tokens:251352 - perf/time_per_step:41.70409296452999 - perf/throughput:753.3792912537954 (TaskRunner pid=2049544) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2049544) "np.float64(0.6828125), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2049544) "np.float64(0.17979486326514935), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2049544) "np.float64(0.7532), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2049544) "np.float64(0.1476854115376502), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6133875), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2049544) "np.float64(0.15413287282151064), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6832499999999999), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2049544) "np.float64(0.17928347494256552), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2049544) "np.float64(0.807675), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2049544) "np.float64(0.11309093981217563), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2049544) "np.float64(0.5515125000000001), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2049544) "np.float64(0.12241577016075818), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 85.21s/it] (TaskRunner pid=2049544) "np.float64(0.7018125), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2049544) "np.float64(0.13581544160753714), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2049544) "np.float64(0.8526375000000002), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2049544) "np.float64(0.0888708326383256), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2049544) "np.float64(0.4995375), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2049544) "np.float64(0.08946281937139845), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2049544) "np.float64(0.710175), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2049544) "np.float64(0.08949529688241312), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "np.float64(0.8910875), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2049544) 'np.float64(0.061136881006381635), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9859375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2049544) 'np.float64(0.03383512807052093), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.99745), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2049544) 'np.float64(0.014287439995541685), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2049544) 'np.float64(0.9741250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2049544) 'np.float64(0.042452979144002284), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9857875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2049544) 'np.float64(0.03387729307844336), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998374999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2049544) 'np.float64(0.002749370152057837), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.9560875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2049544) 'np.float64(0.04764508939236297), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9946999999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2049544) 'np.float64(0.021483511676811038), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.933025), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.0436722700672832), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2049544) 'np.float64(0.9988624999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2049544) 'np.float64(0.009237722502874987), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.9139250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2049544) 'np.float64(0.029568343249240757), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998750000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2049544) "np.float64(0.0017543234207183663), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2049544) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2049544) 'np.float64(2.0)}') (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 49.28s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) wandb: updating run metadata (TaskRunner pid=2049544) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2049544) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2049544) wandb: uploading history steps 98-99, summary, console lines 919-1044 (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run history: (TaskRunner pid=2049544) wandb: actor/entropy ▁▁▂▁▂▁▁▁▂▃▃▂▂▂▃▂▃▃▃▃▄▄▃▄▅▄▆▄▇▆▅▄▆▆▆▅▇█▇▇ (TaskRunner pid=2049544) wandb: actor/grad_norm █▄▇▃█▆▃▇▆▁▄▄▃▃▃▂▁▂▁▂▄▃▂▂▂▂▄▂▃▃▂▃▂▃▂▁▃▂▂▁ (TaskRunner pid=2049544) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2049544) wandb: actor/lr ▁▂▃▄▆███████████████████████████████████ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac ▂▄▅▃▆▇▄▃█▃▃▄▂▂▃▃▂▂▂▂▃▃▃▂▂▁▁▁▁▂▁▁▁▂▂▂▁▁▂▁ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▆▁▁▄▁▃▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁ (TaskRunner pid=2049544) wandb: actor/pg_loss ▆▂▅▅█▆▅▃▃▅▅▄▃▄▅▄▄▃▂▅▁▂▄▄▁▄▆▃▅▄▅▅▂▅▅▄▅▃▅▃ (TaskRunner pid=2049544) wandb: actor/ppo_kl ▁▂▂▃▄▅▃▁▄▃▄▂▄▁▂▂▃▃▂▂▇▄▂▃▂▂▄▄▃▁▄▃▁▂█▃▃▂▃▂ (TaskRunner pid=2049544) wandb: critic/advantages/max █████████▅█████████████▅███▁████████████ (TaskRunner pid=2049544) wandb: critic/advantages/mean ▅▇▆▅▂▄▃▅▅▅▃▂▃▆▃▇▆▂█▂▄▅▃▃▅▁▅▄▄▄▄▅▄▆▆▅▄▃▂▅ (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run summary: (TaskRunner pid=2049544) wandb: actor/entropy 0.37954 (TaskRunner pid=2049544) wandb: actor/grad_norm 0.12032 (TaskRunner pid=2049544) wandb: actor/kl_loss 0 (TaskRunner pid=2049544) wandb: actor/lr 0.0 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac 0.00016 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2049544) wandb: actor/pg_loss -7e-05 (TaskRunner pid=2049544) wandb: actor/ppo_kl -0.00013 (TaskRunner pid=2049544) wandb: critic/advantages/max 0.875 (TaskRunner pid=2049544) wandb: critic/advantages/mean -0.00182 (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: 🚀 View run qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/o8ivivjg (TaskRunner pid=2049544) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2049544) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2049544) wandb: Find logs at: ./wandb/run-20260702_073403-o8ivivjg/logs main_ppo exit code: 0 [2026-07-02 08:56:17] Finished physics grpo [2026-07-02 08:56:17] Starting physics rlsd_tr Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B 2026-07-02 08:56:19,506 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 08:56:29,254 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2074004) TaskRunner hostname: mole-workspace-rw, PID: 2074004 (TaskRunner pid=2074004) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'calculate_entropy': False, (TaskRunner pid=2074004) 'calculate_sum_pi_squared': False, (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra']}, (TaskRunner pid=2074004) 'clip_ratio': 0.2, (TaskRunner pid=2074004) 'clip_ratio_c': 3.0, (TaskRunner pid=2074004) 'clip_ratio_high': 0.28, (TaskRunner pid=2074004) 'clip_ratio_low': 0.2, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_coeff': 0, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'freeze_vision_tower': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'kl_loss_coef': 0.001, (TaskRunner pid=2074004) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'loss_scale_factor': None, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-06, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': 10, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2074004) 'clip_cov_lb': 1.0, (TaskRunner pid=2074004) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'clip_cov_ub': 5.0, (TaskRunner pid=2074004) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'loss_mode': 'rlsd', (TaskRunner pid=2074004) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2074004) 'alpha': 0.5, (TaskRunner pid=2074004) 'distillation_add_tail': True, (TaskRunner pid=2074004) 'distillation_topk': 100, (TaskRunner pid=2074004) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2074004) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2074004) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'loss_weight': 0.0, (TaskRunner pid=2074004) 'mask': 'all'}, (TaskRunner pid=2074004) 'feedback_template': '\n' (TaskRunner pid=2074004) 'The ' (TaskRunner pid=2074004) 'following ' (TaskRunner pid=2074004) 'is ' (TaskRunner pid=2074004) 'feedback ' (TaskRunner pid=2074004) 'from ' (TaskRunner pid=2074004) 'your ' (TaskRunner pid=2074004) 'unsuccessful ' (TaskRunner pid=2074004) 'earlier ' (TaskRunner pid=2074004) 'attempt:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{feedback_raw}', (TaskRunner pid=2074004) 'full_logit_distillation': True, (TaskRunner pid=2074004) 'include_environment_feedback': True, (TaskRunner pid=2074004) 'is_clip': 2, (TaskRunner pid=2074004) 'max_reprompt_len': 10240, (TaskRunner pid=2074004) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2074004) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) 'Correctly ' (TaskRunner pid=2074004) 'solve ' (TaskRunner pid=2074004) 'the ' (TaskRunner pid=2074004) 'original ' (TaskRunner pid=2074004) 'question.', (TaskRunner pid=2074004) 'reprompt_truncation': 'right', (TaskRunner pid=2074004) 'solution_template': '\n' (TaskRunner pid=2074004) 'Correct ' (TaskRunner pid=2074004) 'solution:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{successful_previous_attempt}', (TaskRunner pid=2074004) 'srpo_dynamic_weighting': False, (TaskRunner pid=2074004) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2074004) 'success_reward_threshold': 0.5, (TaskRunner pid=2074004) 'teacher_regularization': 'trust-region', (TaskRunner pid=2074004) 'teacher_update_rate': 0.1, (TaskRunner pid=2074004) 'token_reweight_decay_steps': 0, (TaskRunner pid=2074004) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2074004) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2074004) 'tau_neg': 1.05, (TaskRunner pid=2074004) 'tau_pos': 1.0, (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_kl_loss': False, (TaskRunner pid=2074004) 'use_prefix_grouper': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'hybrid_engine': True, (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2074004) 'custom_chat_template': None, (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'exclude_modules': None, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2074004) 'hf_config_path': None, (TaskRunner pid=2074004) 'lora_adapter_path': None, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2074004) 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': None, (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_liger': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'nccl_timeout': 600, (TaskRunner pid=2074004) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': True, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2074004) 'agent_loop_config_path': None, (TaskRunner pid=2074004) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2074004) 'name': None, (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2074004) 'num_workers': 8}, (TaskRunner pid=2074004) 'calculate_log_probs': True, (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enable_rollout_routing_replay': False, (TaskRunner pid=2074004) 'enforce_eager': False, (TaskRunner pid=2074004) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2074004) 'ignore_eos': False, (TaskRunner pid=2074004) 'layered_summon': False, (TaskRunner pid=2074004) 'load_format': 'dummy', (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'max_num_batched_tokens': 10240, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'mode': 'async', (TaskRunner pid=2074004) 'multi_stage_wake_up': False, (TaskRunner pid=2074004) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'format': 'hermes', (TaskRunner pid=2074004) 'interaction_config_path': None, (TaskRunner pid=2074004) 'max_assistant_turns': None, (TaskRunner pid=2074004) 'max_parallel_calls': 1, (TaskRunner pid=2074004) 'max_tool_response_length': 256, (TaskRunner pid=2074004) 'max_user_turns': None, (TaskRunner pid=2074004) 'num_repeat_rollouts': None, (TaskRunner pid=2074004) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2074004) 'use_inference_chat_template': False}, (TaskRunner pid=2074004) 'n': 8, (TaskRunner pid=2074004) 'name': 'vllm', (TaskRunner pid=2074004) 'over_sample_rate': 0, (TaskRunner pid=2074004) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2074004) 'profiler': (TaskRunner pid=2074004) {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2074004) 'port': 9090, (TaskRunner pid=2074004) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'quantization': None, (TaskRunner pid=2074004) 'quantization_config_file': None, (TaskRunner pid=2074004) 'response_length': 8192, (TaskRunner pid=2074004) 'scheduling_policy': 'fcfs', (TaskRunner pid=2074004) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2074004) 'skip_rollout': False, (TaskRunner pid=2074004) 'skip_tokenizer_init': True, (TaskRunner pid=2074004) 'temperature': 1.0, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 1.0, (TaskRunner pid=2074004) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2074004) 'backend': None, (TaskRunner pid=2074004) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2074004) 'token2text': False}, (TaskRunner pid=2074004) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2074004) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'n': 16, (TaskRunner pid=2074004) 'temperature': 0.6, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 0.95}}}, (TaskRunner pid=2074004) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2074004) 'adv_estimator': 'grpo', (TaskRunner pid=2074004) 'gamma': 1.0, (TaskRunner pid=2074004) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2074004) 'horizon': 10000, (TaskRunner pid=2074004) 'kl_coef': 0.001, (TaskRunner pid=2074004) 'target_kl': 0.1, (TaskRunner pid=2074004) 'type': 'fixed'}, (TaskRunner pid=2074004) 'kl_penalty': 'kl', (TaskRunner pid=2074004) 'lam': 1.0, (TaskRunner pid=2074004) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2074004) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2074004) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2074004) 'loss_type': 'ppo_clip', (TaskRunner pid=2074004) 'rollout_is': 'token', (TaskRunner pid=2074004) 'rollout_is_batch_normalize': False, (TaskRunner pid=2074004) 'rollout_is_threshold': 2.0, (TaskRunner pid=2074004) 'rollout_rs': None, (TaskRunner pid=2074004) 'rollout_rs_threshold': None}, (TaskRunner pid=2074004) 'use_kl_in_reward': False, (TaskRunner pid=2074004) 'use_pf_ppo': False}, (TaskRunner pid=2074004) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2074004) 'cliprange_value': 0.5, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'enable': None, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'forward_micro_batch_size': None, (TaskRunner pid=2074004) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-05, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': -1, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False}, (TaskRunner pid=2074004) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2074004) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2074004) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2074004) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'dataloader_num_workers': 8, (TaskRunner pid=2074004) 'filter_overlong_prompts': True, (TaskRunner pid=2074004) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2074004) 'image_key': 'images', (TaskRunner pid=2074004) 'image_patch_size': 14, (TaskRunner pid=2074004) 'max_prompt_length': 2048, (TaskRunner pid=2074004) 'max_response_length': 8192, (TaskRunner pid=2074004) 'prompt_key': 'prompt', (TaskRunner pid=2074004) 'return_full_prompt': False, (TaskRunner pid=2074004) 'return_multi_modal_inputs': True, (TaskRunner pid=2074004) 'return_raw_chat': True, (TaskRunner pid=2074004) 'return_raw_input_ids': False, (TaskRunner pid=2074004) 'reward_fn_key': 'data_source', (TaskRunner pid=2074004) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2074004) 'seed': None, (TaskRunner pid=2074004) 'shuffle': True, (TaskRunner pid=2074004) 'tokenizer': None, (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'train_batch_size': 32, (TaskRunner pid=2074004) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2074004) 'train_max_samples': 3200, (TaskRunner pid=2074004) 'truncation': 'error', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_shm': False, (TaskRunner pid=2074004) 'val_batch_size': None, (TaskRunner pid=2074004) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2074004) 'val_max_samples': -1, (TaskRunner pid=2074004) 'validation_shuffle': False, (TaskRunner pid=2074004) 'video_key': 'videos'}, (TaskRunner pid=2074004) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2074004) 'capture-range-end': None, (TaskRunner pid=2074004) 'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'kill': 'none', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2074004) 'torch_memory': {'context': 'all', (TaskRunner pid=2074004) 'kw_args': {}, (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'stacks': 'all', (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2074004) 'profile_continuous_steps': False, (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'steps': None, (TaskRunner pid=2074004) 'tool': None}, (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', (TaskRunner pid=2074004) 'include_dashboard': False, (TaskRunner pid=2074004) 'num_cpus': None, (TaskRunner pid=2074004) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'USER': 'root'}}}, (TaskRunner pid=2074004) 'timeline_json_file': None}, (TaskRunner pid=2074004) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2074004) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2074004) 'name': 'custom_reward_manager', (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'name': 'naive', (TaskRunner pid=2074004) 'source': 'register'}, (TaskRunner pid=2074004) 'reward_model': {'enable': False, (TaskRunner pid=2074004) 'enable_resource_pool': False, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'launch_reward_fn_async': False, (TaskRunner pid=2074004) 'max_length': None, (TaskRunner pid=2074004) 'micro_batch_size': None, (TaskRunner pid=2074004) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'model': {'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2074004) 'trust_remote_code': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 0, (TaskRunner pid=2074004) 'num_workers': 1, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'reward_loop_class_name': None, (TaskRunner pid=2074004) 'reward_loop_module_path': None, (TaskRunner pid=2074004) 'reward_loop_source': 'register', (TaskRunner pid=2074004) 'reward_manager': 'naive', (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enforce_eager': True, (TaskRunner pid=2074004) 'engine_kwargs': {}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2074004) 'limit_images': None, (TaskRunner pid=2074004) 'load_format': 'auto', (TaskRunner pid=2074004) 'max_model_len': None, (TaskRunner pid=2074004) 'max_num_batched_tokens': 8192, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'name': '???', (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'response_length': 2048, (TaskRunner pid=2074004) 'skip_tokenizer_init': False, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2074004) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2074004) 'memory_limit_mb': 1024, (TaskRunner pid=2074004) 'url': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_reward_loop': False}, (TaskRunner pid=2074004) 'trainer': {'balance_batch': True, (TaskRunner pid=2074004) 'critic_warmup': 0, (TaskRunner pid=2074004) 'default_hdfs_dir': None, (TaskRunner pid=2074004) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'del_local_ckpt_after_load': False, (TaskRunner pid=2074004) 'device': 'cuda', (TaskRunner pid=2074004) 'esi_redundant_time': 0, (TaskRunner pid=2074004) 'experiment_name': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2074004) 'log_val_generations': 0, (TaskRunner pid=2074004) 'logger': ['console', 'wandb'], (TaskRunner pid=2074004) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2074004) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 1, (TaskRunner pid=2074004) 'project_name': 'SDPO-root', (TaskRunner pid=2074004) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2074004) 'resume_from_path': None, (TaskRunner pid=2074004) 'resume_mode': 'auto', (TaskRunner pid=2074004) 'rollout_data_dir': None, (TaskRunner pid=2074004) 'save_freq': 10, (TaskRunner pid=2074004) 'test_freq': 10, (TaskRunner pid=2074004) 'total_epochs': 30, (TaskRunner pid=2074004) 'total_training_steps': 100, (TaskRunner pid=2074004) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2074004) 'val_before_train': False, (TaskRunner pid=2074004) 'val_only': False, (TaskRunner pid=2074004) 'validation_data_dir': None}, (TaskRunner pid=2074004) 'transfer_queue': {'enable': False}, (TaskRunner pid=2074004) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'dir': '/users/root/SDPO', (TaskRunner pid=2074004) 'log_dir': '/users/root/output', (TaskRunner pid=2074004) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2074004) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) use_critic=need_critic(config), (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2074004) Using dataset class: RLHFDataset (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2074004) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2074004) dataset len: 720 (TaskRunner pid=2074004) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) WARNING:2026-07-02 08:56:38,173:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2074004) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 128.23 examples/s] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) self.use_critic = need_critic(self.config) (WorkerDict pid=2074388) [W702 08:56:47.781903761 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074386) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074387) [W702 08:56:47.990315263 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2074386) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074385) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2074385) Actor use_remove_padding=True (WorkerDict pid=2074385) Actor use_fused_kernels=False (WorkerDict pid=2074385) Actor use_prefix_grouper=False (WorkerDict pid=2074385) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074385) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2074385) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074387) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.99s/it] [repeated 12x across cluster] (WorkerDict pid=2074385) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2074387) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 4x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074385) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2074386) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2074391) warnings.warn( (WorkerDict pid=2074385) reference model: Qwen/Qwen3-4B (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2074391) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2074385) Model config after override: Qwen3Config { (WorkerDict pid=2074385) "architectures": [ (WorkerDict pid=2074385) "Qwen3ForCausalLM" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "attention_bias": false, (WorkerDict pid=2074385) "attention_dropout": 0.0, (WorkerDict pid=2074385) "dtype": "bfloat16", (WorkerDict pid=2074385) "eos_token_id": 151645, (WorkerDict pid=2074385) "head_dim": 128, (WorkerDict pid=2074385) "hidden_act": "silu", (WorkerDict pid=2074385) "hidden_size": 2560, (WorkerDict pid=2074385) "initializer_range": 0.02, (WorkerDict pid=2074385) "intermediate_size": 9728, (WorkerDict pid=2074385) "layer_types": [ (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "max_position_embeddings": 40960, (WorkerDict pid=2074385) "max_window_layers": 36, (WorkerDict pid=2074385) "model_type": "qwen3", (WorkerDict pid=2074385) "num_attention_heads": 32, (WorkerDict pid=2074385) "num_hidden_layers": 36, (WorkerDict pid=2074385) "num_key_value_heads": 8, (WorkerDict pid=2074385) "pad_token_id": 151643, (WorkerDict pid=2074385) "rms_norm_eps": 1e-06, (WorkerDict pid=2074385) "rope_scaling": null, (WorkerDict pid=2074385) "rope_theta": 1000000, (WorkerDict pid=2074385) "sliding_window": null, (WorkerDict pid=2074385) "tie_word_embeddings": true, (WorkerDict pid=2074385) "transformers_version": "4.57.1", (WorkerDict pid=2074385) "use_cache": true, (WorkerDict pid=2074385) "use_sliding_window": false, (WorkerDict pid=2074385) "vocab_size": 151936 (WorkerDict pid=2074385) } (WorkerDict pid=2074385) (WorkerDict pid=2074387) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) Ref use_remove_padding=True (WorkerDict pid=2074385) Ref use_fused_kernels=False (WorkerDict pid=2074385) Ref use_prefix_grouper=False (WorkerDict pid=2074392) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074392) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074392) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:06<00:03, 3.02s/it] [repeated 14x across cluster] (WorkerDict pid=2074392) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.04s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2075282) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2075282) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2075282) ['serve', (vLLMHttpServer pid=2075282) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2075282) '--dtype', (vLLMHttpServer pid=2075282) 'bfloat16', (vLLMHttpServer pid=2075282) '--load_format', (vLLMHttpServer pid=2075282) 'dummy', (vLLMHttpServer pid=2075282) '--trust_remote_code', (vLLMHttpServer pid=2075282) '--max_model_len', (vLLMHttpServer pid=2075282) '40960', (vLLMHttpServer pid=2075282) '--max_num_seqs', (vLLMHttpServer pid=2075282) '1024', (vLLMHttpServer pid=2075282) '--enable_chunked_prefill', (vLLMHttpServer pid=2075282) '--max_num_batched_tokens', (vLLMHttpServer pid=2075282) '10240', (vLLMHttpServer pid=2075282) '--enable_prefix_caching', (vLLMHttpServer pid=2075282) '--enable_sleep_mode', (vLLMHttpServer pid=2075282) '--logprobs_mode', (vLLMHttpServer pid=2075282) 'processed_logprobs', (vLLMHttpServer pid=2075282) '--disable_custom_all_reduce', (vLLMHttpServer pid=2075282) '--gpu_memory_utilization', (vLLMHttpServer pid=2075282) '0.8', (vLLMHttpServer pid=2075282) '--disable_log_stats', (vLLMHttpServer pid=2075282) '--tensor_parallel_size', (vLLMHttpServer pid=2075282) '2', (vLLMHttpServer pid=2075282) '--seed', (vLLMHttpServer pid=2075282) '0', (vLLMHttpServer pid=2075282) '--override_generation_config', (vLLMHttpServer pid=2075282) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2075282) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2075282) '--hf_overrides', (vLLMHttpServer pid=2075282) '{}', (vLLMHttpServer pid=2075282) '--scheduling_policy', (vLLMHttpServer pid=2075282) 'fcfs'] (vLLMHttpServer pid=2075282) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2075282) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2075282) WARNING 07-02 08:57:45 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2074385) WARNING 07-02 08:57:52 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2075287) WARNING 07-02 08:57:46 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2074389) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/aotautograd/aqwmczjqsm4pvwfimkpe3rswyj2xfwbwedbijxvwd7tfo5n2eqnk/.2074388.133380852872896.tmp' (vLLMHttpServer pid=2075287) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2075287) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2075287) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2075287) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2074385) 2026-07-02 08:58:08,146 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2074386) 2026-07-02 08:58:08,165 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2074004) step:10 - global_seqlen/min:14569 - global_seqlen/max:25213 - global_seqlen/minmax_diff:10644 - global_seqlen/balanced_min:21068 - global_seqlen/balanced_max:21078 - global_seqlen/mean:21074.625 - actor/entropy:0.13031111657619476 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5310845375061035 - training/rollout_probs_diff_mean:0.0036879268009215593 - training/rollout_probs_diff_std:0.013145294040441513 - training/rollout_actor_probs_pearson_corr:0.9974448680877686 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999222755432129 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00015043777239043266 - rollout_corr/rollout_is_max:1.8829189538955688 - rollout_corr/rollout_is_min:0.005527450703084469 - rollout_corr/rollout_is_std:0.03613611310720444 - rollout_corr/rollout_is_eff_sample_size:0.9986955276850646 - rollout_corr/rollout_is_seq_mean:0.9999170899391174 - rollout_corr/rollout_is_seq_std:0.002527824370190501 - rollout_corr/rollout_is_seq_max:1.0131255388259888 - rollout_corr/rollout_is_seq_min:0.9898325800895691 - rollout_corr/rollout_is_seq_max_deviation:0.01312553882598877 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1374189392663538) - rollout_corr/training_log_ppl:np.float64(0.12561230757273734) - rollout_corr/kl:np.float64(0.0010259306004876123) - rollout_corr/k3_kl:np.float64(0.001012646021507635) - rollout_corr/rollout_ppl:np.float64(1.1362624615430832) - rollout_corr/rollout_log_ppl:np.float64(0.1245863765507238) - rollout_corr/log_ppl_diff:np.float64(0.0010259310220135376) - rollout_corr/log_ppl_abs_diff:np.float64(0.00206260806589853) - rollout_corr/log_ppl_diff_max:np.float64(0.03187315911054611) - rollout_corr/log_ppl_diff_min:np.float64(-0.010174185037612915) - rollout_corr/ppl_ratio:np.float64(1.0010317743290216) - rollout_corr/chi2_token:np.float64(0.0018072936218231916) - rollout_corr/chi2_seq:np.float64(0.3196559448260814) - actor/pg_loss:np.float64(0.0017917423974722624) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004963221986145072) - actor/ppo_kl:np.float64(8.269873148769591e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(95332.59919399978) - self_distillation/token_reweight_w_std:np.float64(1506740.680497941) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996795877814293) - self_distillation/token_reweight_w_clip_frac:np.float64(0.033084604699979536) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.3074006289243698) - perf/mfu/actor:np.float64(0.03828182711170613) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.1017189025879) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.596875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10804651530040861) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7887749999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.019017006085694965) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6468625) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07892990828797705) - val-aux/sciknoweval/score/mean@16:np.float64(0.596875) - val-aux/sciknoweval/score/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10804651530040861) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7887749999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.019017006085694965) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6468625) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07892990828797705) - val-core/sciknoweval/acc/mean@16:np.float64(0.596875) - val-aux/sciknoweval/acc/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10804651530040861) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7887749999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.019017006085694965) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6468625) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07892990828797705) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02228859585888569) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0055870721336611015) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9884875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.02914679194594295) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9939625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.021812516180988194) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9776375000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03766353637974691) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.997625) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.013472998070955988) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9620000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04262616078525806) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9992875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005053486156825662) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9403500000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03977205801345875) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998875) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0015086398882979818) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032043245155364275 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032043245155364275 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:389.48828125 - response_length/max:1190.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:389.48828125 - response_length_non_aborted/max:1190.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015878118574619293 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1143263150006533) - timing_s/agent_loop/generate_sequences/max:np.float64(8.482053555548191) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.887139186161221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.482053555548191) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:337 - timing_s/agent_loop/slowest/response_length:1190 - timing_s/gen:14.678978031501174 - timing_s/reward:0.05808144994080067 - timing_s/old_log_prob:2.4217102844268084 - timing_s/adv:0.5421234201639891 - timing_s/update_actor:15.863357232883573 - timing_s/step:33.64816468022764 - timing_s/testing:93.59819382615387 - timing_s/save_checkpoint:6.729555023834109 - timing_s/stop_profile:0.00014853663742542267 - timing_per_token_ms/update_actor:0.09409038851749184 - timing_per_token_ms/adv:0.003215498616013269 - timing_per_token_ms/gen:0.14721818523404281 - perf/total_num_tokens:168597 - perf/time_per_step:33.64816468022764 - perf/throughput:626.3231650308669 (TaskRunner pid=2074004) Training Progress: 10%|█ | 10/100 [08:42<1:43:00, 68.67s/it] (TaskRunner pid=2074004) step:11 - global_seqlen/min:15931 - global_seqlen/max:27826 - global_seqlen/minmax_diff:11895 - global_seqlen/balanced_min:22821 - global_seqlen/balanced_max:22841 - global_seqlen/mean:22835.0 - actor/entropy:0.12203283607959747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40049630403518677 - training/rollout_probs_diff_mean:0.003223797306418419 - training/rollout_probs_diff_std:0.012214891612529755 - training/rollout_actor_probs_pearson_corr:0.9976435303688049 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9997804164886475 - rollout_corr/rollout_is_ratio_fraction_high:2.6688492653192952e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.3376985306385905e-05 - rollout_corr/rollout_is_max:2.5879318714141846 - rollout_corr/rollout_is_min:0.06322140991687775 - rollout_corr/rollout_is_std:0.033195044845342636 - rollout_corr/rollout_is_eff_sample_size:0.9988987665423175 - rollout_corr/rollout_is_seq_mean:0.9999120235443115 - rollout_corr/rollout_is_seq_std:0.002345130778849125 - rollout_corr/rollout_is_seq_max:1.0110429525375366 - rollout_corr/rollout_is_seq_min:0.9912058711051941 - rollout_corr/rollout_is_seq_max_deviation:0.011042952537536621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1257328824140131) - rollout_corr/training_log_ppl:np.float64(0.1140785860188771) - rollout_corr/kl:np.float64(0.0006342849212974677) - rollout_corr/k3_kl:np.float64(0.0007503947783931153) - rollout_corr/rollout_ppl:np.float64(1.1249900460243225) - rollout_corr/rollout_log_ppl:np.float64(0.11344430113604176) - rollout_corr/log_ppl_diff:np.float64(0.0006342848828353453) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019007355404028203) - rollout_corr/log_ppl_diff_max:np.float64(0.010488316416740417) - rollout_corr/log_ppl_diff_min:np.float64(-0.008857212960720062) - rollout_corr/ppl_ratio:np.float64(1.0006378127727658) - rollout_corr/chi2_token:np.float64(0.001916841370984912) - rollout_corr/chi2_seq:np.float64(0.37116734171286225) - actor/pg_loss:np.float64(0.0027762840036302805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005819873397285846) - actor/ppo_kl:np.float64(-7.081770549888233e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(90827.28597010626) - self_distillation/token_reweight_w_std:np.float64(1824642.9953269707) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001720192376524) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04453263468167279) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.29148364439606667) - perf/mfu/actor:np.float64(0.04050194858164669) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.13693618774414) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004592644691001624 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004592644691001624 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.09375 - response_length/max:1273.0 - response_length/min:68.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.09375 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.3513795137405396e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6711478792130947) - timing_s/agent_loop/generate_sequences/max:np.float64(9.547774223610759) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.419399234910088) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.547774223610759) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:15.524424999952316 - timing_s/reward:0.057088837027549744 - timing_s/old_log_prob:2.659148082137108 - timing_s/adv:0.5871537029743195 - timing_s/update_actor:16.28799468278885 - timing_s/step:35.222145142033696 - timing_s/stop_profile:0.0001278892159461975 - timing_per_token_ms/update_actor:0.0891613459754152 - timing_per_token_ms/adv:0.003214110482670897 - timing_per_token_ms/gen:0.13810783040310579 - perf/total_num_tokens:182680 - perf/time_per_step:35.222145142033696 - perf/throughput:648.3137216066087 (TaskRunner pid=2074004) Training Progress: 11%|█ | 11/100 [09:18<1:26:42, 58.45s/it] (TaskRunner pid=2074004) step:12 - global_seqlen/min:16305 - global_seqlen/max:30523 - global_seqlen/minmax_diff:14218 - global_seqlen/balanced_min:22309 - global_seqlen/balanced_max:28829 - global_seqlen/mean:23127.125 - actor/entropy:0.12043065577745438 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6972464323043823 - training/rollout_probs_diff_mean:0.0032291633542627096 - training/rollout_probs_diff_std:0.012314102612435818 - training/rollout_actor_probs_pearson_corr:0.9975891709327698 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999233484268188 - rollout_corr/rollout_is_ratio_fraction_high:6.790191400796175e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5463217752985656e-05 - rollout_corr/rollout_is_max:3.958061456680298 - rollout_corr/rollout_is_min:0.3131382167339325 - rollout_corr/rollout_is_std:0.033765628933906555 - rollout_corr/rollout_is_eff_sample_size:0.9988609428668661 - rollout_corr/rollout_is_seq_mean:0.9999712705612183 - rollout_corr/rollout_is_seq_std:0.0021552243269979954 - rollout_corr/rollout_is_seq_max:1.0135027170181274 - rollout_corr/rollout_is_seq_min:0.9933554530143738 - rollout_corr/rollout_is_seq_max_deviation:0.013502717018127441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1240018154494464) - rollout_corr/training_log_ppl:np.float64(0.11423524462952628) - rollout_corr/kl:np.float64(0.0006360744373044014) - rollout_corr/k3_kl:np.float64(0.0007582616624119964) - rollout_corr/rollout_ppl:np.float64(1.1232725959271193) - rollout_corr/rollout_log_ppl:np.float64(0.11359916946094017) - rollout_corr/log_ppl_diff:np.float64(0.0006360751685861032) - rollout_corr/log_ppl_abs_diff:np.float64(0.001725419224385405) - rollout_corr/log_ppl_diff_max:np.float64(0.009453028440475464) - rollout_corr/log_ppl_diff_min:np.float64(-0.006760995835065842) - rollout_corr/ppl_ratio:np.float64(1.0006386556196958) - rollout_corr/chi2_token:np.float64(0.0019189040176570415) - rollout_corr/chi2_seq:np.float64(1.096296020084992) - actor/pg_loss:np.float64(0.0028759860433638096) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007010218323557638) - actor/ppo_kl:np.float64(-5.775291747767142e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(54891.713903531665) - self_distillation/token_reweight_w_std:np.float64(995585.2390469867) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996340507641435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05258868928649463) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.3390292413532734) - perf/mfu/actor:np.float64(0.03925409114627591) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.01633071899414) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022134963423013687 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022134963423013687 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:460.22265625 - response_length/max:8192.0 - response_length/min:78.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:460.22265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.100325405597687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941336028277874) - timing_s/agent_loop/generate_sequences/max:np.float64(50.94393623247743) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.578305393028131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.94393623247743) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.6255786344409 - timing_s/reward:0.06343746557831764 - timing_s/old_log_prob:2.6843459140509367 - timing_s/adv:0.540890708565712 - timing_s/update_actor:17.096599811688066 - timing_s/step:77.09979971125722 - timing_s/stop_profile:0.00011315196752548218 - timing_per_token_ms/update_actor:0.09240556171426445 - timing_per_token_ms/adv:0.002923464917092548 - timing_per_token_ms/gen:0.480623158240669 - perf/total_num_tokens:185017 - perf/time_per_step:77.09979971125722 - perf/throughput:299.96349000402455 (TaskRunner pid=2074004) Training Progress: 12%|█▏ | 12/100 [10:35<1:34:04, 64.14s/it] (TaskRunner pid=2074004) step:13 - global_seqlen/min:18538 - global_seqlen/max:25997 - global_seqlen/minmax_diff:7459 - global_seqlen/balanced_min:22090 - global_seqlen/balanced_max:22116 - global_seqlen/mean:22109.375 - actor/entropy:0.14604653418064117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7997847199440002 - training/rollout_probs_diff_mean:0.0036600856110453606 - training/rollout_probs_diff_std:0.012734896503388882 - training/rollout_actor_probs_pearson_corr:0.9978328943252563 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:0.9999634623527527 - rollout_corr/rollout_is_ratio_fraction_high:2.843844413291663e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.531533239874989e-05 - rollout_corr/rollout_is_max:2.8094775676727295 - rollout_corr/rollout_is_min:6.210129504324868e-05 - rollout_corr/rollout_is_std:0.03579304739832878 - rollout_corr/rollout_is_eff_sample_size:0.9987203780861295 - rollout_corr/rollout_is_seq_mean:0.9999745488166809 - rollout_corr/rollout_is_seq_std:0.002158660674467683 - rollout_corr/rollout_is_seq_max:1.0080207586288452 - rollout_corr/rollout_is_seq_min:0.9927275776863098 - rollout_corr/rollout_is_seq_max_deviation:0.008020758628845215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1676474194973707) - rollout_corr/training_log_ppl:np.float64(0.14873290639661718) - rollout_corr/kl:np.float64(0.0009227445474575546) - rollout_corr/k3_kl:np.float64(0.0009155499404727152) - rollout_corr/rollout_ppl:np.float64(1.1665417086333036) - rollout_corr/rollout_log_ppl:np.float64(0.14781016217602883) - rollout_corr/log_ppl_diff:np.float64(0.0009227442205883563) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018867802718887106) - rollout_corr/log_ppl_diff_max:np.float64(0.060229599475860596) - rollout_corr/log_ppl_diff_min:np.float64(-0.004808485507965088) - rollout_corr/ppl_ratio:np.float64(1.0009324494749308) - rollout_corr/chi2_token:np.float64(0.0014858830254524946) - rollout_corr/chi2_seq:np.float64(0.4379577168729156) - actor/pg_loss:np.float64(0.002721810364164412) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005266812750051031) - actor/ppo_kl:np.float64(1.293608784180833e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(196224.12132495665) - self_distillation/token_reweight_w_std:np.float64(2690730.0869071595) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001536855706945) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06831683132622857) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.4452543556690216) - perf/mfu/actor:np.float64(0.03950919290459659) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.9030246734619) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021506573539227247 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021506573539227247 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.07421875 - response_length/max:1343.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.07421875 - response_length_non_aborted/max:1343.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001416429877281189 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2486017309129238) - timing_s/agent_loop/generate_sequences/max:np.float64(9.4201366789639) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.31274351340835) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.4201366789639) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1343 - timing_s/gen:15.389248048886657 - timing_s/reward:0.059297624975442886 - timing_s/old_log_prob:2.4455661196261644 - timing_s/adv:0.5371290631592274 - timing_s/update_actor:16.235428664833307 - timing_s/step:34.75727551802993 - timing_s/stop_profile:3.674998879432678e-05 - timing_per_token_ms/update_actor:0.09179040941248513 - timing_per_token_ms/adv:0.003036772088532734 - timing_per_token_ms/gen:0.14588209467050892 - perf/total_num_tokens:176875 - perf/time_per_step:34.75727551802993 - perf/throughput:636.107826936292 (TaskRunner pid=2074004) Training Progress: 13%|█▎ | 13/100 [11:09<1:20:06, 55.24s/it] (TaskRunner pid=2074004) step:14 - global_seqlen/min:15836 - global_seqlen/max:27136 - global_seqlen/minmax_diff:11300 - global_seqlen/balanced_min:21152 - global_seqlen/balanced_max:21214 - global_seqlen/mean:21164.375 - actor/entropy:0.13442379236221313 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967105984687805 - training/rollout_probs_diff_mean:0.003770587034523487 - training/rollout_probs_diff_std:0.013384245336055756 - training/rollout_actor_probs_pearson_corr:0.9973688125610352 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001509189605713 - rollout_corr/rollout_is_ratio_fraction_high:3.0175318897818215e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012070127559127286 - rollout_corr/rollout_is_max:3.7234184741973877 - rollout_corr/rollout_is_min:0.041150957345962524 - rollout_corr/rollout_is_std:0.03739259019494057 - rollout_corr/rollout_is_eff_sample_size:0.9986041031225663 - rollout_corr/rollout_is_seq_mean:1.000020146369934 - rollout_corr/rollout_is_seq_std:0.002401510952040553 - rollout_corr/rollout_is_seq_max:1.0093960762023926 - rollout_corr/rollout_is_seq_min:0.9911119341850281 - rollout_corr/rollout_is_seq_max_deviation:0.009396076202392578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1523799053393304) - rollout_corr/training_log_ppl:np.float64(0.13728270668070763) - rollout_corr/kl:np.float64(0.0008854879360526979) - rollout_corr/k3_kl:np.float64(0.0008427418540151166) - rollout_corr/rollout_ppl:np.float64(1.1513059278950095) - rollout_corr/rollout_log_ppl:np.float64(0.13639721802610438) - rollout_corr/log_ppl_diff:np.float64(0.0008854886546032503) - rollout_corr/log_ppl_abs_diff:np.float64(0.001983832713449374) - rollout_corr/log_ppl_diff_max:np.float64(0.011519268155097961) - rollout_corr/log_ppl_diff_min:np.float64(-0.00734923779964447) - rollout_corr/ppl_ratio:np.float64(1.0008891641627997) - rollout_corr/chi2_token:np.float64(0.001656870823353529) - rollout_corr/chi2_seq:np.float64(1.6609382745809853) - actor/pg_loss:np.float64(0.004066150169819593) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009369107503971463) - actor/ppo_kl:np.float64(0.00011594514723167748) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(220119.85480736103) - self_distillation/token_reweight_w_std:np.float64(3045461.0947162565) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001637001987547) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08081251988187432) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.4899965673685074) - perf/mfu/actor:np.float64(0.03798268804792023) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.0521469116211) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005462990142405033 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005462990142405033 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.35546875 - response_length/max:1228.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:388.35546875 - response_length_non_aborted/max:1228.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.705312550067902e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7120892163366079) - timing_s/agent_loop/generate_sequences/max:np.float64(8.587548367679119) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.029178587690694) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.587548367679119) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1228 - timing_s/gen:14.909624475985765 - timing_s/reward:0.05932668782770634 - timing_s/old_log_prob:2.404671125113964 - timing_s/adv:0.5401296932250261 - timing_s/update_actor:16.007855907082558 - timing_s/step:34.01097614504397 - timing_s/stop_profile:0.00011675432324409485 - timing_per_token_ms/update_actor:0.09454481828002574 - timing_per_token_ms/adv:0.003190087666332139 - timing_per_token_ms/gen:0.14996755626173836 - perf/total_num_tokens:169315 - perf/time_per_step:34.01097614504397 - perf/throughput:622.2807281314695 (TaskRunner pid=2074004) Training Progress: 14%|█▍ | 14/100 [11:43<1:10:00, 48.85s/it] (TaskRunner pid=2074004) step:15 - global_seqlen/min:13000 - global_seqlen/max:27017 - global_seqlen/minmax_diff:14017 - global_seqlen/balanced_min:21238 - global_seqlen/balanced_max:21268 - global_seqlen/mean:21253.875 - actor/entropy:0.11480943113565445 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772899866104126 - training/rollout_probs_diff_mean:0.0029744282364845276 - training/rollout_probs_diff_std:0.01138862781226635 - training/rollout_actor_probs_pearson_corr:0.9978798031806946 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999694228172302 - rollout_corr/rollout_is_ratio_fraction_high:2.9455948606482707e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.9455948606482707e-05 - rollout_corr/rollout_is_max:2.554089307785034 - rollout_corr/rollout_is_min:0.44323766231536865 - rollout_corr/rollout_is_std:0.032444074749946594 - rollout_corr/rollout_is_eff_sample_size:0.998948250895511 - rollout_corr/rollout_is_seq_mean:1.000088095664978 - rollout_corr/rollout_is_seq_std:0.001991908298805356 - rollout_corr/rollout_is_seq_max:1.0099899768829346 - rollout_corr/rollout_is_seq_min:0.9942845106124878 - rollout_corr/rollout_is_seq_max_deviation:0.00998997688293457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.126635740045458) - rollout_corr/training_log_ppl:np.float64(0.11509814600140089) - rollout_corr/kl:np.float64(0.0005166397006295398) - rollout_corr/k3_kl:np.float64(0.0006518633542142993) - rollout_corr/rollout_ppl:np.float64(1.126022334676236) - rollout_corr/rollout_log_ppl:np.float64(0.11458150647194998) - rollout_corr/log_ppl_diff:np.float64(0.000516639529450913) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016528939577256097) - rollout_corr/log_ppl_diff_max:np.float64(0.007337689399719238) - rollout_corr/log_ppl_diff_min:np.float64(-0.009172983467578888) - rollout_corr/ppl_ratio:np.float64(1.0005192304961383) - rollout_corr/chi2_token:np.float64(0.0016447578091174364) - rollout_corr/chi2_seq:np.float64(0.39781012921594083) - actor/pg_loss:np.float64(0.002455260371789336) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006204967883149948) - actor/ppo_kl:np.float64(4.5773536659687863e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(192128.32348678238) - self_distillation/token_reweight_w_std:np.float64(3233259.402355935) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002948227338493) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06099457883101422) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.3918348103761673) - perf/mfu/actor:np.float64(0.03843189897453754) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.9206657409668) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015439826529473066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015439826529473066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:397.83984375 - response_length/max:1327.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:397.83984375 - response_length_non_aborted/max:1327.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.262189269065857e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7463704198598862) - timing_s/agent_loop/generate_sequences/max:np.float64(8.988874156028032) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.964175615350541) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.988874156028032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1327 - timing_s/gen:14.802029374986887 - timing_s/reward:0.06032639183104038 - timing_s/old_log_prob:2.3973333202302456 - timing_s/adv:0.5653208959847689 - timing_s/update_actor:15.675101241096854 - timing_s/step:33.58521797135472 - timing_s/stop_profile:3.232806921005249e-05 - timing_per_token_ms/update_actor:0.09218966683191214 - timing_per_token_ms/adv:0.0033248107461861004 - timing_per_token_ms/gen:0.14533593895732705 - perf/total_num_tokens:170031 - perf/time_per_step:33.58521797135472 - perf/throughput:632.8342134961789 (TaskRunner pid=2074004) Training Progress: 15%|█▌ | 15/100 [12:17<1:02:41, 44.25s/it] (TaskRunner pid=2074004) step:16 - global_seqlen/min:16611 - global_seqlen/max:28011 - global_seqlen/minmax_diff:11400 - global_seqlen/balanced_min:20244 - global_seqlen/balanced_max:26855 - global_seqlen/mean:21153.75 - actor/entropy:0.12396589666604996 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8040750622749329 - training/rollout_probs_diff_mean:0.0032445986289530993 - training/rollout_probs_diff_std:0.012475800700485706 - training/rollout_actor_probs_pearson_corr:0.9975688457489014 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9998238682746887 - rollout_corr/rollout_is_ratio_fraction_high:3.8920348742976785e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.730087185744196e-05 - rollout_corr/rollout_is_max:2.290397882461548 - rollout_corr/rollout_is_min:3.1829338240640936e-06 - rollout_corr/rollout_is_std:0.03366485983133316 - rollout_corr/rollout_is_eff_sample_size:0.9988676034321022 - rollout_corr/rollout_is_seq_mean:0.9998279809951782 - rollout_corr/rollout_is_seq_std:0.002493756590411067 - rollout_corr/rollout_is_seq_max:1.0092169046401978 - rollout_corr/rollout_is_seq_min:0.9886294007301331 - rollout_corr/rollout_is_seq_max_deviation:0.011370599269866943 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1432740236632526) - rollout_corr/training_log_ppl:np.float64(0.12944419973246113) - rollout_corr/kl:np.float64(0.00073082422317583) - rollout_corr/k3_kl:np.float64(0.00073855356768604) - rollout_corr/rollout_ppl:np.float64(1.1423929436132312) - rollout_corr/rollout_log_ppl:np.float64(0.1287133757614356) - rollout_corr/log_ppl_diff:np.float64(0.0007308239710255293) - rollout_corr/log_ppl_abs_diff:np.float64(0.001978938538741204) - rollout_corr/log_ppl_diff_max:np.float64(0.010752767324447632) - rollout_corr/log_ppl_diff_min:np.float64(-0.0069452449679374695) - rollout_corr/ppl_ratio:np.float64(1.000734378118068) - rollout_corr/chi2_token:np.float64(0.001494420226663351) - rollout_corr/chi2_seq:np.float64(0.7517771443817765) - actor/pg_loss:np.float64(0.002636288059875369) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006587515040337166) - actor/ppo_kl:np.float64(-0.0003202963638955225) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(219386.07391313673) - self_distillation/token_reweight_w_std:np.float64(3319235.12259777) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016510419081897) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05755699300789274) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.43999863415956497) - perf/mfu/actor:np.float64(0.03636736138421752) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.8370819091797) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04519382491707802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04519382491707802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:401.4609375 - response_length/max:8192.0 - response_length/min:97.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:401.4609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.450495958328247e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.158904004842043) - timing_s/agent_loop/generate_sequences/max:np.float64(50.02395389974117) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.92313740587997) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.02395389974117) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.74822269193828 - timing_s/reward:0.05993708409368992 - timing_s/old_log_prob:2.6834859494119883 - timing_s/adv:0.5374118015170097 - timing_s/update_actor:16.872027203440666 - timing_s/step:75.98889152519405 - timing_s/stop_profile:0.00010964088141918182 - timing_per_token_ms/update_actor:0.09969879574213003 - timing_per_token_ms/adv:0.003175629625462446 - timing_per_token_ms/gen:0.5424350778595587 - perf/total_num_tokens:169230 - perf/time_per_step:75.98889152519405 - perf/throughput:278.37950489100757 (TaskRunner pid=2074004) Training Progress: 16%|█▌ | 16/100 [13:33<1:15:20, 53.82s/it] (TaskRunner pid=2074004) step:17 - global_seqlen/min:16383 - global_seqlen/max:22557 - global_seqlen/minmax_diff:6174 - global_seqlen/balanced_min:19789 - global_seqlen/balanced_max:19793 - global_seqlen/mean:19791.0 - actor/entropy:0.1373070627450943 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535119295120239 - training/rollout_probs_diff_mean:0.0037673222832381725 - training/rollout_probs_diff_std:0.013155417516827583 - training/rollout_actor_probs_pearson_corr:0.997542679309845 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.546680768020451e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.765353075228631e-05 - rollout_corr/rollout_is_max:2.7404403686523438 - rollout_corr/rollout_is_min:0.32417601346969604 - rollout_corr/rollout_is_std:0.037032198160886765 - rollout_corr/rollout_is_eff_sample_size:0.9986302566846117 - rollout_corr/rollout_is_seq_mean:1.0000929832458496 - rollout_corr/rollout_is_seq_std:0.0029761020559817553 - rollout_corr/rollout_is_seq_max:1.0141609907150269 - rollout_corr/rollout_is_seq_min:0.9883509874343872 - rollout_corr/rollout_is_seq_max_deviation:0.014160990715026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1427205740474164) - rollout_corr/training_log_ppl:np.float64(0.13067101137858117) - rollout_corr/kl:np.float64(0.001165712662167806) - rollout_corr/k3_kl:np.float64(0.0011212053367444241) - rollout_corr/rollout_ppl:np.float64(1.1413688631728292) - rollout_corr/rollout_log_ppl:np.float64(0.12950529891531914) - rollout_corr/log_ppl_diff:np.float64(0.0011657124632620253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574236517830286) - rollout_corr/log_ppl_diff_max:np.float64(0.018865570425987244) - rollout_corr/log_ppl_diff_min:np.float64(-0.009756885468959808) - rollout_corr/ppl_ratio:np.float64(1.0011729013640434) - rollout_corr/chi2_token:np.float64(0.001890765968710184) - rollout_corr/chi2_seq:np.float64(1.113262691302225) - actor/pg_loss:np.float64(0.004973054630681872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013291486598063784) - actor/ppo_kl:np.float64(0.000477771908172997) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(430373.50576436426) - self_distillation/token_reweight_w_std:np.float64(5256699.274065455) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005029244348407) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08690081464010291) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.691158652305603) - perf/mfu/actor:np.float64(0.03633210791443406) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.84893798828125) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020605919416993856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020605919416993856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.125 - response_length/max:919.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.125 - response_length_non_aborted/max:919.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.005889296531677e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8571333661675453) - timing_s/agent_loop/generate_sequences/max:np.float64(7.130504677072167) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.622363231239433) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.130504677072167) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:919 - timing_s/gen:12.924050455912948 - timing_s/reward:0.059431292116642 - timing_s/old_log_prob:2.4258709643036127 - timing_s/adv:0.542781762778759 - timing_s/update_actor:15.836449747905135 - timing_s/step:31.876996461302042 - timing_s/stop_profile:0.0001156199723482132 - timing_per_token_ms/update_actor:0.10002305181588307 - timing_per_token_ms/adv:0.003428210820440851 - timing_per_token_ms/gen:0.14337116675444786 - perf/total_num_tokens:158328 - perf/time_per_step:31.876996461302042 - perf/throughput:620.8552309508152 (TaskRunner pid=2074004) Training Progress: 17%|█▋ | 17/100 [14:05<1:05:20, 47.23s/it] (TaskRunner pid=2074004) step:18 - global_seqlen/min:15614 - global_seqlen/max:27961 - global_seqlen/minmax_diff:12347 - global_seqlen/balanced_min:22588 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22607.75 - actor/entropy:0.11930368840694427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7529500722885132 - training/rollout_probs_diff_mean:0.0031947942916303873 - training/rollout_probs_diff_std:0.01253251638263464 - training/rollout_actor_probs_pearson_corr:0.9974825382232666 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999328255653381 - rollout_corr/rollout_is_ratio_fraction_high:3.530138565110974e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015885622997302562 - rollout_corr/rollout_is_max:2.3141205310821533 - rollout_corr/rollout_is_min:0.1902111917734146 - rollout_corr/rollout_is_std:0.03377268835902214 - rollout_corr/rollout_is_eff_sample_size:0.998860586053378 - rollout_corr/rollout_is_seq_mean:1.000009298324585 - rollout_corr/rollout_is_seq_std:0.0019673723727464676 - rollout_corr/rollout_is_seq_max:1.007437825202942 - rollout_corr/rollout_is_seq_min:0.9938141703605652 - rollout_corr/rollout_is_seq_max_deviation:0.0074378252029418945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.129554508253932) - rollout_corr/training_log_ppl:np.float64(0.11883334071171703) - rollout_corr/kl:np.float64(0.0006264003223446224) - rollout_corr/k3_kl:np.float64(0.0007624520011759728) - rollout_corr/rollout_ppl:np.float64(1.1288568419404328) - rollout_corr/rollout_log_ppl:np.float64(0.1182069395945291) - rollout_corr/log_ppl_diff:np.float64(0.0006264011171879247) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016409209492849186) - rollout_corr/log_ppl_diff_max:np.float64(0.008434589952230453) - rollout_corr/log_ppl_diff_min:np.float64(-0.006128266453742981) - rollout_corr/ppl_ratio:np.float64(1.0006289719603956) - rollout_corr/chi2_token:np.float64(0.0018175377044826746) - rollout_corr/chi2_seq:np.float64(0.6837903573177755) - actor/pg_loss:np.float64(0.004627972492016852) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010802133826928184) - actor/ppo_kl:np.float64(-1.8316878190205443e-05) - actor/pg_clipfrac_lower:np.float64(1.7377642052451847e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.390625) - self_distillation/token_reweight_w_mean:np.float64(205932.63544678479) - self_distillation/token_reweight_w_std:np.float64(3275553.812427452) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015711192972958) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07417638991319109) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.49010825902223587) - perf/mfu/actor:np.float64(0.04100974753741407) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.18482971191406) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013242431916296482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013242431916296482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.6171875 - response_length/max:1530.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:442.6171875 - response_length_non_aborted/max:1530.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014469213783740997 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7575890514999628) - timing_s/agent_loop/generate_sequences/max:np.float64(10.56420186534524) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3792500576455495) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.56420186534524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1530 - timing_s/gen:16.62187496200204 - timing_s/reward:0.0622586477547884 - timing_s/old_log_prob:2.5925257727503777 - timing_s/adv:0.5586317237466574 - timing_s/update_actor:15.944112984463573 - timing_s/step:35.868158884346485 - timing_s/stop_profile:0.00011105090379714966 - timing_per_token_ms/update_actor:0.08815623505470233 - timing_per_token_ms/adv:0.0030887180488253885 - timing_per_token_ms/gen:0.14669380427148565 - perf/total_num_tokens:180862 - perf/time_per_step:35.868158884346485 - perf/throughput:630.3013788049889 (TaskRunner pid=2074004) Training Progress: 18%|█▊ | 18/100 [14:41<59:54, 43.83s/it] (TaskRunner pid=2074004) step:19 - global_seqlen/min:16296 - global_seqlen/max:29019 - global_seqlen/minmax_diff:12723 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21199 - global_seqlen/mean:21195.375 - actor/entropy:0.10421572625637054 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5834612846374512 - training/rollout_probs_diff_mean:0.002985131461173296 - training/rollout_probs_diff_std:0.011980867013335228 - training/rollout_actor_probs_pearson_corr:0.9974684119224548 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.953125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.953125 - rollout_corr/rollout_is_mean:1.0000929832458496 - rollout_corr/rollout_is_ratio_fraction_high:3.9236074371729046e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010789919906528667 - rollout_corr/rollout_is_max:3.10373592376709 - rollout_corr/rollout_is_min:0.06492556631565094 - rollout_corr/rollout_is_std:0.033075422048568726 - rollout_corr/rollout_is_eff_sample_size:0.9989074497400088 - rollout_corr/rollout_is_seq_mean:1.0000252723693848 - rollout_corr/rollout_is_seq_std:0.0022536139003932476 - rollout_corr/rollout_is_seq_max:1.0099234580993652 - rollout_corr/rollout_is_seq_min:0.9871556758880615 - rollout_corr/rollout_is_seq_max_deviation:0.012844324111938477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1145465495064855) - rollout_corr/training_log_ppl:np.float64(0.10453275293184561) - rollout_corr/kl:np.float64(0.0007910890604869714) - rollout_corr/k3_kl:np.float64(0.0006869278773535825) - rollout_corr/rollout_ppl:np.float64(1.11366256326437) - rollout_corr/rollout_log_ppl:np.float64(0.10374166368637816) - rollout_corr/log_ppl_diff:np.float64(0.0007910892454674467) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018341713002882898) - rollout_corr/log_ppl_diff_max:np.float64(0.020886391401290894) - rollout_corr/log_ppl_diff_min:np.float64(-0.008934102952480316) - rollout_corr/ppl_ratio:np.float64(1.0007949848659337) - rollout_corr/chi2_token:np.float64(0.0011295359581708908) - rollout_corr/chi2_seq:np.float64(0.5068342476151884) - actor/pg_loss:np.float64(0.004318527295254171) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010698111041165248) - actor/ppo_kl:np.float64(0.00017291589237800054) - actor/pg_clipfrac_lower:np.float64(1.0065717106044758e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(318408.1419896148) - self_distillation/token_reweight_w_std:np.float64(4400530.746190617) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.003058377886191) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06988180117332377) - self_distillation/empty_target_batch:np.float64(0.046875) - actor/grad_norm:np.float64(0.41591396927833557) - perf/mfu/actor:np.float64(0.03846046736330761) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.08876419067383) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0076731047593057156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0076731047593057156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.23046875 - response_length/max:1107.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.23046875 - response_length_non_aborted/max:1107.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.310990571975708e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9409778341650963) - timing_s/agent_loop/generate_sequences/max:np.float64(8.202672267332673) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.057653636649775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.202672267332673) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:282 - timing_s/agent_loop/slowest/response_length:1064 - timing_s/gen:14.027731182053685 - timing_s/reward:0.059779345989227295 - timing_s/old_log_prob:2.6110658906400204 - timing_s/adv:0.565576946362853 - timing_s/update_actor:15.789859525859356 - timing_s/step:33.141286708414555 - timing_s/stop_profile:0.00012139417231082916 - timing_per_token_ms/update_actor:0.09312090211814698 - timing_per_token_ms/adv:0.003335497404285446 - timing_per_token_ms/gen:0.13759827343672384 - perf/total_num_tokens:169563 - perf/time_per_step:33.141286708414555 - perf/throughput:639.5459291150124 (TaskRunner pid=2074004) Training Progress: 19%|█▉ | 19/100 [15:14<54:51, 40.64s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We need to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2074004) step:20 - global_seqlen/min:16557 - global_seqlen/max:25344 - global_seqlen/minmax_diff:8787 - global_seqlen/balanced_min:20112 - global_seqlen/balanced_max:20126 - global_seqlen/mean:20123.125 - actor/entropy:0.1016768291592598 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7593052387237549 - training/rollout_probs_diff_mean:0.0031040834728628397 - training/rollout_probs_diff_std:0.012476732954382896 - training/rollout_actor_probs_pearson_corr:0.9970778822898865 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9998236298561096 - rollout_corr/rollout_is_ratio_fraction_high:1.0585034942778293e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012702042295131832 - rollout_corr/rollout_is_max:3.1519858837127686 - rollout_corr/rollout_is_min:0.22677943110466003 - rollout_corr/rollout_is_std:0.0336710549890995 - rollout_corr/rollout_is_eff_sample_size:0.9988670087351662 - rollout_corr/rollout_is_seq_mean:0.9997695684432983 - rollout_corr/rollout_is_seq_std:0.00212579732760787 - rollout_corr/rollout_is_seq_max:1.0059138536453247 - rollout_corr/rollout_is_seq_min:0.9923778176307678 - rollout_corr/rollout_is_seq_max_deviation:0.007622182369232178 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1031243042089045) - rollout_corr/training_log_ppl:np.float64(0.09565349744298146) - rollout_corr/kl:np.float64(0.0009412075490131677) - rollout_corr/k3_kl:np.float64(0.0007612467196338457) - rollout_corr/rollout_ppl:np.float64(1.1020435886457562) - rollout_corr/rollout_log_ppl:np.float64(0.09471228851907654) - rollout_corr/log_ppl_diff:np.float64(0.0009412089239049237) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019292085235065315) - rollout_corr/log_ppl_diff_max:np.float64(0.011152833700180054) - rollout_corr/log_ppl_diff_min:np.float64(-0.005067870020866394) - rollout_corr/ppl_ratio:np.float64(1.000944792991504) - rollout_corr/chi2_token:np.float64(0.001142266672104597) - rollout_corr/chi2_seq:np.float64(0.5473281410522759) - actor/pg_loss:np.float64(0.0028149266727268696) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000719407254791804) - actor/ppo_kl:np.float64(9.32751950681876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(257358.8287920449) - self_distillation/token_reweight_w_std:np.float64(3547259.863764613) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000823934096843) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04804125138616655) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4549396261572838) - perf/mfu/actor:np.float64(0.03647682227242746) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.2781219482422) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6) - val-aux/sciknoweval/reward/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.11172766129349468) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8170249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0340747398319288) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6269750000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07494075368453937) - val-aux/sciknoweval/score/mean@16:np.float64(0.6) - val-aux/sciknoweval/score/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.11172766129349468) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8170249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0340747398319288) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6269750000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07494075368453937) - val-core/sciknoweval/acc/mean@16:np.float64(0.6) - val-aux/sciknoweval/acc/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.11172766129349468) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8170249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0340747398319288) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6269750000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07494075368453937) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.024206145913796356) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995875000000002) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006276698746323264) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9878875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03259636494557759) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9940750000000002) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.023588209728692643) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9772874999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04188475149905875) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9979250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01420822710607692) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9603625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0489078128065511) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0053028861809800276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9365375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04813723029580622) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023961872793734074 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023961872793734074 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.03515625 - response_length/max:1235.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.03515625 - response_length_non_aborted/max:1235.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.98199325799942e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1489538233727217) - timing_s/agent_loop/generate_sequences/max:np.float64(8.91213802434504) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7332966142930673) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.91213802434504) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1235 - timing_s/gen:14.779135162010789 - timing_s/reward:0.05472232215106487 - timing_s/old_log_prob:2.653603931888938 - timing_s/adv:0.5555432084947824 - timing_s/update_actor:15.782804861664772 - timing_s/step:33.897329518571496 - timing_s/testing:88.8644837513566 - timing_s/save_checkpoint:6.838149785995483 - timing_s/stop_profile:0.00017092563211917877 - timing_per_token_ms/update_actor:0.09803897792753842 - timing_per_token_ms/adv:0.0034509004472142277 - timing_per_token_ms/gen:0.15643766115197769 - perf/total_num_tokens:160985 - perf/time_per_step:33.897329518571496 - perf/throughput:593.649272252407 (TaskRunner pid=2074004) Training Progress: 20%|██ | 20/100 [17:24<1:29:48, 67.36s/it] (TaskRunner pid=2074004) step:21 - global_seqlen/min:18517 - global_seqlen/max:25867 - global_seqlen/minmax_diff:7350 - global_seqlen/balanced_min:21107 - global_seqlen/balanced_max:21412 - global_seqlen/mean:21162.875 - actor/entropy:0.09419602900743484 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948364734649658 - training/rollout_probs_diff_mean:0.002783035859465599 - training/rollout_probs_diff_std:0.011772817932069302 - training/rollout_actor_probs_pearson_corr:0.9972338080406189 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000512599945068 - rollout_corr/rollout_is_ratio_fraction_high:1.9861960026901215e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012910272926092148 - rollout_corr/rollout_is_max:2.5543158054351807 - rollout_corr/rollout_is_min:0.1329210251569748 - rollout_corr/rollout_is_std:0.03238707408308983 - rollout_corr/rollout_is_eff_sample_size:0.9989522955064933 - rollout_corr/rollout_is_seq_mean:1.0000442266464233 - rollout_corr/rollout_is_seq_std:0.0020267271902412176 - rollout_corr/rollout_is_seq_max:1.0177290439605713 - rollout_corr/rollout_is_seq_min:0.9942728877067566 - rollout_corr/rollout_is_seq_max_deviation:0.01772904396057129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0939804124645889) - rollout_corr/training_log_ppl:np.float64(0.08795880044999649) - rollout_corr/kl:np.float64(0.0004990964827804589) - rollout_corr/k3_kl:np.float64(0.0006357246575703357) - rollout_corr/rollout_ppl:np.float64(1.093429901637137) - rollout_corr/rollout_log_ppl:np.float64(0.08745970321979257) - rollout_corr/log_ppl_diff:np.float64(0.0004990972302039154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015424156808876432) - rollout_corr/log_ppl_diff_max:np.float64(0.005901060998439789) - rollout_corr/log_ppl_diff_min:np.float64(-0.0117645263671875) - rollout_corr/ppl_ratio:np.float64(1.000501276459545) - rollout_corr/chi2_token:np.float64(0.0017355745658278465) - rollout_corr/chi2_seq:np.float64(0.9698769934475422) - actor/pg_loss:np.float64(0.0027807088335976005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005495589271049539) - actor/ppo_kl:np.float64(-4.407575040854539e-05) - actor/pg_clipfrac_lower:np.float64(1.0252624633722007e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(141096.08209011448) - self_distillation/token_reweight_w_std:np.float64(1977869.731875727) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000101735349745) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04118481201294344) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.3234409838914871) - perf/mfu/actor:np.float64(0.03855656297649644) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.51013565063477) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002361090388149023 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002361090388149023 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:393.33984375 - response_length/max:1529.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:393.33984375 - response_length_non_aborted/max:1529.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015509501099586487 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4505607336759567) - timing_s/agent_loop/generate_sequences/max:np.float64(10.29461688362062) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.021393945564341) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.29461688362062) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1529 - timing_s/gen:15.94972218759358 - timing_s/reward:0.0574874933809042 - timing_s/old_log_prob:2.583099478855729 - timing_s/adv:0.6336547415703535 - timing_s/update_actor:15.871118316426873 - timing_s/step:35.191847467795014 - timing_s/stop_profile:0.00011814944446086884 - timing_per_token_ms/update_actor:0.09374386937282195 - timing_per_token_ms/adv:0.003742726009405347 - timing_per_token_ms/gen:0.15839636712442107 - perf/total_num_tokens:169303 - perf/time_per_step:35.191847467795014 - perf/throughput:601.3573177528319 (TaskRunner pid=2074004) Training Progress: 21%|██ | 21/100 [17:59<1:16:00, 57.72s/it] (TaskRunner pid=2074004) step:22 - global_seqlen/min:15901 - global_seqlen/max:25415 - global_seqlen/minmax_diff:9514 - global_seqlen/balanced_min:20195 - global_seqlen/balanced_max:20210 - global_seqlen/mean:20203.0 - actor/entropy:0.10593846440315247 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5313484072685242 - training/rollout_probs_diff_mean:0.003251798916608095 - training/rollout_probs_diff_std:0.012875033542513847 - training/rollout_actor_probs_pearson_corr:0.9971089959144592 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.00010347366333 - rollout_corr/rollout_is_ratio_fraction_high:4.197447924525477e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010493619629414752 - rollout_corr/rollout_is_max:2.9541661739349365 - rollout_corr/rollout_is_min:0.09866257756948471 - rollout_corr/rollout_is_std:0.03540460020303726 - rollout_corr/rollout_is_eff_sample_size:0.9987482024906889 - rollout_corr/rollout_is_seq_mean:1.0001643896102905 - rollout_corr/rollout_is_seq_std:0.0024298520293086767 - rollout_corr/rollout_is_seq_max:1.0121618509292603 - rollout_corr/rollout_is_seq_min:0.9910787343978882 - rollout_corr/rollout_is_seq_max_deviation:0.012161850929260254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1144308936782181) - rollout_corr/training_log_ppl:np.float64(0.10470682490995387) - rollout_corr/kl:np.float64(0.000816442931922623) - rollout_corr/k3_kl:np.float64(0.0008207170265279728) - rollout_corr/rollout_ppl:np.float64(1.1135047189891338) - rollout_corr/rollout_log_ppl:np.float64(0.10389038186986) - rollout_corr/log_ppl_diff:np.float64(0.0008164430400938727) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018727829810814) - rollout_corr/log_ppl_diff_max:np.float64(0.0158127099275589) - rollout_corr/log_ppl_diff_min:np.float64(-0.007389448583126068) - rollout_corr/ppl_ratio:np.float64(1.0008202025201172) - rollout_corr/chi2_token:np.float64(0.0015751214232295752) - rollout_corr/chi2_seq:np.float64(0.42517587705515325) - actor/pg_loss:np.float64(0.0036053532967343926) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010071134870486276) - actor/ppo_kl:np.float64(0.00023022429938990285) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(218233.54828879703) - self_distillation/token_reweight_w_std:np.float64(3210707.643729917) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001665785908699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.058951567800249904) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.4596095457673073) - perf/mfu/actor:np.float64(0.036942325232477896) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.46227645874023) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006771008018404245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006771008018404245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.25 - response_length/max:1292.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:372.25 - response_length_non_aborted/max:1292.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001899413764476776 - timing_s/agent_loop/generate_sequences/min:np.float64(1.149434920400381) - timing_s/agent_loop/generate_sequences/max:np.float64(8.853524098172784) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8520722229222883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.853524098172784) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1292 - timing_s/gen:14.650895247235894 - timing_s/reward:0.04885207116603851 - timing_s/old_log_prob:2.5285760313272476 - timing_s/adv:0.5984222404658794 - timing_s/update_actor:15.83981061168015 - timing_s/step:33.70206049270928 - timing_s/stop_profile:0.00015066377818584442 - timing_per_token_ms/update_actor:0.09800407496213527 - timing_per_token_ms/adv:0.0037025580388177462 - timing_per_token_ms/gen:0.15374092561320407 - perf/total_num_tokens:161624 - perf/time_per_step:33.70206049270928 - perf/throughput:599.4588967155432 (TaskRunner pid=2074004) Training Progress: 22%|██▏ | 22/100 [18:33<1:05:41, 50.53s/it] (TaskRunner pid=2074004) step:23 - global_seqlen/min:16336 - global_seqlen/max:30208 - global_seqlen/minmax_diff:13872 - global_seqlen/balanced_min:23332 - global_seqlen/balanced_max:23529 - global_seqlen/mean:23365.625 - actor/entropy:0.13051186501979828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46211716532707214 - training/rollout_probs_diff_mean:0.003518641460686922 - training/rollout_probs_diff_std:0.012740753591060638 - training/rollout_actor_probs_pearson_corr:0.9975020289421082 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999224543571472 - rollout_corr/rollout_is_ratio_fraction_high:2.5563025701558217e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013633613707497716 - rollout_corr/rollout_is_max:3.358520746231079 - rollout_corr/rollout_is_min:0.2698378264904022 - rollout_corr/rollout_is_std:0.03545843064785004 - rollout_corr/rollout_is_eff_sample_size:0.9987440406221719 - rollout_corr/rollout_is_seq_mean:0.9998674988746643 - rollout_corr/rollout_is_seq_std:0.002211816841736436 - rollout_corr/rollout_is_seq_max:1.0098694562911987 - rollout_corr/rollout_is_seq_min:0.9933016896247864 - rollout_corr/rollout_is_seq_max_deviation:0.00986945629119873 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1375553528778255) - rollout_corr/training_log_ppl:np.float64(0.12625176343863131) - rollout_corr/kl:np.float64(0.0009684273467929216) - rollout_corr/k3_kl:np.float64(0.0008128368172322098) - rollout_corr/rollout_ppl:np.float64(1.1364499349147081) - rollout_corr/rollout_log_ppl:np.float64(0.12528333482623566) - rollout_corr/log_ppl_diff:np.float64(0.0009684286123956554) - rollout_corr/log_ppl_abs_diff:np.float64(0.001832397589168977) - rollout_corr/log_ppl_diff_max:np.float64(0.011887356638908386) - rollout_corr/log_ppl_diff_min:np.float64(-0.008548229932785034) - rollout_corr/ppl_ratio:np.float64(1.000971759436652) - rollout_corr/chi2_token:np.float64(0.0012992981355637312) - rollout_corr/chi2_seq:np.float64(0.5191449248231947) - actor/pg_loss:np.float64(0.005724000162445009) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016155378229996131) - actor/ppo_kl:np.float64(0.00011277476574633738) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(304747.9913651231) - self_distillation/token_reweight_w_std:np.float64(4644227.323464029) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019670443143696) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08922295937372837) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5448018610477448) - perf/mfu/actor:np.float64(0.04195048253218733) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.12269973754883) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004490571562200785 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004490571562200785 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.42578125 - response_length/max:1832.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:458.42578125 - response_length_non_aborted/max:1832.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005329102277755737 - timing_s/agent_loop/generate_sequences/min:np.float64(1.116574076935649) - timing_s/agent_loop/generate_sequences/max:np.float64(11.822160206735134) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7243306759992265) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.822160206735134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1832 - timing_s/gen:18.11809141561389 - timing_s/reward:0.05629480630159378 - timing_s/old_log_prob:2.732456060126424 - timing_s/adv:0.6987331956624985 - timing_s/update_actor:15.840048659592867 - timing_s/step:37.55766446888447 - timing_s/stop_profile:6.579048931598663e-05 - timing_per_token_ms/update_actor:0.08474012924752101 - timing_per_token_ms/adv:0.00373804036732646 - timing_per_token_ms/gen:0.154384411800011 - perf/total_num_tokens:186925 - perf/time_per_step:37.55766446888447 - perf/throughput:622.1266772154537 (TaskRunner pid=2074004) Training Progress: 23%|██▎ | 23/100 [19:11<59:59, 46.75s/it] (TaskRunner pid=2074004) step:24 - global_seqlen/min:17135 - global_seqlen/max:25434 - global_seqlen/minmax_diff:8299 - global_seqlen/balanced_min:20734 - global_seqlen/balanced_max:20748 - global_seqlen/mean:20742.75 - actor/entropy:0.10690584033727646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519520163536072 - training/rollout_probs_diff_mean:0.0030989660881459713 - training/rollout_probs_diff_std:0.012050447054207325 - training/rollout_actor_probs_pearson_corr:0.9973185062408447 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0002549886703491 - rollout_corr/rollout_is_ratio_fraction_high:2.0413579477462918e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.165431790985167e-05 - rollout_corr/rollout_is_max:2.1403613090515137 - rollout_corr/rollout_is_min:0.21778209507465363 - rollout_corr/rollout_is_std:0.033476658165454865 - rollout_corr/rollout_is_eff_sample_size:0.9988811627143783 - rollout_corr/rollout_is_seq_mean:1.0003387928009033 - rollout_corr/rollout_is_seq_std:0.0020440113730728626 - rollout_corr/rollout_is_seq_max:1.0078518390655518 - rollout_corr/rollout_is_seq_min:0.9929835200309753 - rollout_corr/rollout_is_seq_max_deviation:0.007851839065551758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.109124246519059) - rollout_corr/training_log_ppl:np.float64(0.10045992501545697) - rollout_corr/kl:np.float64(0.00014452066502457228) - rollout_corr/k3_kl:np.float64(0.0006306728797227379) - rollout_corr/rollout_ppl:np.float64(1.1089712120592594) - rollout_corr/rollout_log_ppl:np.float64(0.10031540389900329) - rollout_corr/log_ppl_diff:np.float64(0.0001445211164536886) - rollout_corr/log_ppl_abs_diff:np.float64(0.001574805770360399) - rollout_corr/log_ppl_diff_max:np.float64(0.011304967105388641) - rollout_corr/log_ppl_diff_min:np.float64(-0.011325351893901825) - rollout_corr/ppl_ratio:np.float64(1.000147320330143) - rollout_corr/chi2_token:np.float64(0.0022659352980554104) - rollout_corr/chi2_seq:np.float64(0.813906122231856) - actor/pg_loss:np.float64(0.002153943874873221) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006025306415722298) - actor/ppo_kl:np.float64(-0.00010819188584676454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(135130.44865711615) - self_distillation/token_reweight_w_std:np.float64(2006584.000496325) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000246322248131) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04379224638978485) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.33335286378860474) - perf/mfu/actor:np.float64(0.037579949707391716) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.17943572998047) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005197807680815458 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005197807680815458 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.7109375 - response_length/max:1139.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.7109375 - response_length_non_aborted/max:1139.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.317960262298584e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8356650900095701) - timing_s/agent_loop/generate_sequences/max:np.float64(8.357717419043183) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8592212262228713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.357717419043183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1139 - timing_s/gen:14.09934650734067 - timing_s/reward:0.05855764448642731 - timing_s/old_log_prob:2.4800929073244333 - timing_s/adv:0.5536916963756084 - timing_s/update_actor:15.75769667699933 - timing_s/step:33.040523858740926 - timing_s/stop_profile:0.0001149345189332962 - timing_per_token_ms/update_actor:0.0949590620638496 - timing_per_token_ms/adv:0.003336657967094578 - timing_per_token_ms/gen:0.1439090626833718 - perf/total_num_tokens:165942 - perf/time_per_step:33.040523858740926 - perf/throughput:627.7972494831516 (TaskRunner pid=2074004) Training Progress: 24%|██▍ | 24/100 [19:44<54:01, 42.65s/it] (TaskRunner pid=2074004) step:25 - global_seqlen/min:15092 - global_seqlen/max:31297 - global_seqlen/minmax_diff:16205 - global_seqlen/balanced_min:21652 - global_seqlen/balanced_max:28332 - global_seqlen/mean:22490.0 - actor/entropy:0.10956007242202759 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190921068191528 - training/rollout_probs_diff_mean:0.0033610411919653416 - training/rollout_probs_diff_std:0.013345292769372463 - training/rollout_actor_probs_pearson_corr:0.9969186782836914 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000109314918518 - rollout_corr/rollout_is_ratio_fraction_high:3.6875877412967384e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.531381248147227e-05 - rollout_corr/rollout_is_max:2.4000637531280518 - rollout_corr/rollout_is_min:0.031893547624349594 - rollout_corr/rollout_is_std:0.03650045394897461 - rollout_corr/rollout_is_eff_sample_size:0.9986697273810201 - rollout_corr/rollout_is_seq_mean:1.0001757144927979 - rollout_corr/rollout_is_seq_std:0.002466355450451374 - rollout_corr/rollout_is_seq_max:1.009514331817627 - rollout_corr/rollout_is_seq_min:0.9879665374755859 - rollout_corr/rollout_is_seq_max_deviation:0.012033462524414062 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1296572694554925) - rollout_corr/training_log_ppl:np.float64(0.11874415549391415) - rollout_corr/kl:np.float64(0.0007365228179097016) - rollout_corr/k3_kl:np.float64(0.000829679726990662) - rollout_corr/rollout_ppl:np.float64(1.1288006519898772) - rollout_corr/rollout_log_ppl:np.float64(0.11800763171049766) - rollout_corr/log_ppl_diff:np.float64(0.0007365237834164873) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019228525488870218) - rollout_corr/log_ppl_diff_max:np.float64(0.01413746178150177) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073609501123428345) - rollout_corr/ppl_ratio:np.float64(1.0007400074973702) - rollout_corr/chi2_token:np.float64(0.001852479763329029) - rollout_corr/chi2_seq:np.float64(0.793041022028774) - actor/pg_loss:np.float64(0.0035626995377242565) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000992866120668623) - actor/ppo_kl:np.float64(0.00013374969253865387) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(210967.00421022926) - self_distillation/token_reweight_w_std:np.float64(2976356.6710397173) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001523660030216) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0665886076312745) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4494774639606476) - perf/mfu/actor:np.float64(0.03848721918082872) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.24299240112305) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.052680645138025284 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.052680645138025284 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.71875 - response_length/max:8192.0 - response_length/min:75.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:423.71875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015040487051010132 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8068552948534489) - timing_s/agent_loop/generate_sequences/max:np.float64(50.265202172100544) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.223772806173656) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.265202172100544) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.000157160684466 - timing_s/reward:0.062263449653983116 - timing_s/old_log_prob:2.6894225608557463 - timing_s/adv:0.575139531865716 - timing_s/update_actor:16.96509013324976 - timing_s/step:76.38079810328782 - timing_s/stop_profile:0.00012892484664916992 - timing_per_token_ms/update_actor:0.09429240847737749 - timing_per_token_ms/adv:0.003196640350520876 - timing_per_token_ms/gen:0.5162637100881745 - perf/total_num_tokens:179920 - perf/time_per_step:76.38079810328782 - perf/throughput:294.44573189176873 (TaskRunner pid=2074004) Training Progress: 25%|██▌ | 25/100 [21:00<1:05:59, 52.79s/it] (TaskRunner pid=2074004) step:26 - global_seqlen/min:17856 - global_seqlen/max:22321 - global_seqlen/minmax_diff:4465 - global_seqlen/balanced_min:19619 - global_seqlen/balanced_max:19629 - global_seqlen/mean:19624.625 - actor/entropy:0.10051669180393219 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30771172046661377 - training/rollout_probs_diff_mean:0.0031875635031610727 - training/rollout_probs_diff_std:0.012896761298179626 - training/rollout_actor_probs_pearson_corr:0.9968799948692322 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0000146627426147 - rollout_corr/rollout_is_ratio_fraction_high:2.208553814853076e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.93849171209149e-05 - rollout_corr/rollout_is_max:2.9205586910247803 - rollout_corr/rollout_is_min:0.3763677477836609 - rollout_corr/rollout_is_std:0.03457672521471977 - rollout_corr/rollout_is_eff_sample_size:0.9988058776680073 - rollout_corr/rollout_is_seq_mean:1.0000602006912231 - rollout_corr/rollout_is_seq_std:0.0020706949289888144 - rollout_corr/rollout_is_seq_max:1.0055534839630127 - rollout_corr/rollout_is_seq_min:0.991748571395874 - rollout_corr/rollout_is_seq_max_deviation:0.008251428604125977 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0977428681217134) - rollout_corr/training_log_ppl:np.float64(0.09058145447852439) - rollout_corr/kl:np.float64(0.0006791098424780984) - rollout_corr/k3_kl:np.float64(0.0007086409321068743) - rollout_corr/rollout_ppl:np.float64(1.0969782588072121) - rollout_corr/rollout_log_ppl:np.float64(0.0899023435304116) - rollout_corr/log_ppl_diff:np.float64(0.0006791109481127933) - rollout_corr/log_ppl_abs_diff:np.float64(0.001920343165693339) - rollout_corr/log_ppl_diff_max:np.float64(0.024022668600082397) - rollout_corr/log_ppl_diff_min:np.float64(-0.005489669740200043) - rollout_corr/ppl_ratio:np.float64(1.0006832026410848) - rollout_corr/chi2_token:np.float64(0.001474759541451931) - rollout_corr/chi2_seq:np.float64(0.6005796948447824) - actor/pg_loss:np.float64(0.003621779615059495) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007727315569354687) - actor/ppo_kl:np.float64(0.000100072484177538) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(201738.92587617878) - self_distillation/token_reweight_w_std:np.float64(2411880.3968902696) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00004991912283) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04766093760554213) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.45207298547029495) - perf/mfu/actor:np.float64(0.03555715209782605) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.1187286376953) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-5.383349707699381e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-5.383349707699381e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:353.73828125 - response_length/max:1273.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:353.73828125 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011607818305492401 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5807047858834267) - timing_s/agent_loop/generate_sequences/max:np.float64(8.703462390229106) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3116236440982902) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.703462390229106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:14.44053604081273 - timing_s/reward:0.058792874217033386 - timing_s/old_log_prob:2.4093342386186123 - timing_s/adv:0.5303562916815281 - timing_s/update_actor:15.99789859354496 - timing_s/step:33.523452650755644 - timing_s/stop_profile:2.9999762773513794e-05 - timing_per_token_ms/update_actor:0.10189939039309642 - timing_per_token_ms/adv:0.0033781301023683773 - timing_per_token_ms/gen:0.15946349857893627 - perf/total_num_tokens:156997 - perf/time_per_step:33.523452650755644 - perf/throughput:585.3998752589001 (TaskRunner pid=2074004) Training Progress: 26%|██▌ | 26/100 [21:34<57:59, 47.02s/it] (TaskRunner pid=2074004) step:27 - global_seqlen/min:19536 - global_seqlen/max:27649 - global_seqlen/minmax_diff:8113 - global_seqlen/balanced_min:24156 - global_seqlen/balanced_max:24592 - global_seqlen/mean:24227.0 - actor/entropy:0.08968006819486618 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35172876715660095 - training/rollout_probs_diff_mean:0.0026838507037609816 - training/rollout_probs_diff_std:0.01177211757749319 - training/rollout_actor_probs_pearson_corr:0.9971114993095398 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.000118613243103 - rollout_corr/rollout_is_ratio_fraction_high:3.249707515351474e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.68698815186508e-05 - rollout_corr/rollout_is_max:9.403421401977539 - rollout_corr/rollout_is_min:0.415858656167984 - rollout_corr/rollout_is_std:0.03169693797826767 - rollout_corr/rollout_is_eff_sample_size:0.9989963125083661 - rollout_corr/rollout_is_seq_mean:1.0000813007354736 - rollout_corr/rollout_is_seq_std:0.0021606753580272198 - rollout_corr/rollout_is_seq_max:1.014573335647583 - rollout_corr/rollout_is_seq_min:0.991974413394928 - rollout_corr/rollout_is_seq_max_deviation:0.014573335647583008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0907162227667868) - rollout_corr/training_log_ppl:np.float64(0.08525840124275419) - rollout_corr/kl:np.float64(0.0006036972056477907) - rollout_corr/k3_kl:np.float64(0.0006888124125836725) - rollout_corr/rollout_ppl:np.float64(1.0900488472543657) - rollout_corr/rollout_log_ppl:np.float64(0.08465470290684607) - rollout_corr/log_ppl_diff:np.float64(0.0006036983359081205) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015368564454547595) - rollout_corr/log_ppl_diff_max:np.float64(0.013522587716579437) - rollout_corr/log_ppl_diff_min:np.float64(-0.0061819590628147125) - rollout_corr/ppl_ratio:np.float64(1.0006064060144126) - rollout_corr/chi2_token:np.float64(0.0026917706709355116) - rollout_corr/chi2_seq:np.float64(5.136045180493966) - actor/pg_loss:np.float64(0.0036772675812244415) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007991504742221878) - actor/ppo_kl:np.float64(0.00012214449816383421) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(157488.76363183185) - self_distillation/token_reweight_w_std:np.float64(2501365.0777522195) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000293314224109) - self_distillation/token_reweight_w_clip_frac:np.float64(0.055393004906363785) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.4007357805967331) - perf/mfu/actor:np.float64(0.043591956964482634) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.22241592407227) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009190579294227064 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009190579294227064 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.8125 - response_length/max:2125.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.8125 - response_length_non_aborted/max:2125.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.861744284629822e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1442433521151543) - timing_s/agent_loop/generate_sequences/max:np.float64(13.537522902712226) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.885370632960985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.537522902712226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:2125 - timing_s/gen:19.567774968221784 - timing_s/reward:0.06262494809925556 - timing_s/old_log_prob:2.4061235673725605 - timing_s/adv:0.5981502141803503 - timing_s/update_actor:16.023667991161346 - timing_s/step:38.744677659124136 - timing_s/stop_profile:0.00011847354471683502 - timing_per_token_ms/update_actor:0.08267463981901053 - timing_per_token_ms/adv:0.0030861756211063604 - timing_per_token_ms/gen:0.15897386396904478 - perf/total_num_tokens:193816 - perf/time_per_step:38.744677659124136 - perf/throughput:625.2987884722971 (TaskRunner pid=2074004) Training Progress: 27%|██▋ | 27/100 [22:13<54:12, 44.55s/it] (TaskRunner pid=2074004) step:28 - global_seqlen/min:15689 - global_seqlen/max:28335 - global_seqlen/minmax_diff:12646 - global_seqlen/balanced_min:22049 - global_seqlen/balanced_max:22067 - global_seqlen/mean:22059.625 - actor/entropy:0.11999335139989853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807188868522644 - training/rollout_probs_diff_mean:0.00325206876732409 - training/rollout_probs_diff_std:0.012448713183403015 - training/rollout_actor_probs_pearson_corr:0.9974731802940369 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001393556594849 - rollout_corr/rollout_is_ratio_fraction_high:5.649025843013078e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.590530392713845e-05 - rollout_corr/rollout_is_max:3.7063121795654297 - rollout_corr/rollout_is_min:0.3865959644317627 - rollout_corr/rollout_is_std:0.03557758033275604 - rollout_corr/rollout_is_eff_sample_size:0.998736192621627 - rollout_corr/rollout_is_seq_mean:1.0002413988113403 - rollout_corr/rollout_is_seq_std:0.002391271060332656 - rollout_corr/rollout_is_seq_max:1.0125643014907837 - rollout_corr/rollout_is_seq_min:0.9939977526664734 - rollout_corr/rollout_is_seq_max_deviation:0.012564301490783691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1245770892128348) - rollout_corr/training_log_ppl:np.float64(0.1141112526966026) - rollout_corr/kl:np.float64(0.0005681062639624201) - rollout_corr/k3_kl:np.float64(0.0007897823702087692) - rollout_corr/rollout_ppl:np.float64(1.1239187237806618) - rollout_corr/rollout_log_ppl:np.float64(0.11354314523669018) - rollout_corr/log_ppl_diff:np.float64(0.0005681074599124258) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016994898305711104) - rollout_corr/log_ppl_diff_max:np.float64(0.01184430718421936) - rollout_corr/log_ppl_diff_min:np.float64(-0.006551269441843033) - rollout_corr/ppl_ratio:np.float64(1.0005709801334888) - rollout_corr/chi2_token:np.float64(0.002181260846555233) - rollout_corr/chi2_seq:np.float64(0.662524172803387) - actor/pg_loss:np.float64(0.002658500336110592) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007556931734598038) - actor/ppo_kl:np.float64(8.9224213090211e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(108684.6212300947) - self_distillation/token_reweight_w_std:np.float64(1726174.4731562997) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000196180306375) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04746657198120374) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3256382420659065) - perf/mfu/actor:np.float64(0.039955600042607634) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.3211898803711) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012464340776205063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012464340776205063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.89453125 - response_length/max:1399.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.89453125 - response_length_non_aborted/max:1399.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016648322343826294 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5349066201597452) - timing_s/agent_loop/generate_sequences/max:np.float64(9.329830139875412) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.712864897781401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.329830139875412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1399 - timing_s/gen:15.155654072761536 - timing_s/reward:0.061105452477931976 - timing_s/old_log_prob:2.4498131908476353 - timing_s/adv:0.594944417476654 - timing_s/update_actor:15.982728907838464 - timing_s/step:34.33417861536145 - timing_s/stop_profile:0.00012039020657539368 - timing_per_token_ms/update_actor:0.09056550659767824 - timing_per_token_ms/adv:0.003371229211039705 - timing_per_token_ms/gen:0.14269114018775042 - perf/total_num_tokens:176477 - perf/time_per_step:34.33417861536145 - perf/throughput:642.4975313121457 (TaskRunner pid=2074004) Training Progress: 28%|██▊ | 28/100 [22:47<49:47, 41.50s/it] (TaskRunner pid=2074004) step:29 - global_seqlen/min:18998 - global_seqlen/max:31893 - global_seqlen/minmax_diff:12895 - global_seqlen/balanced_min:22424 - global_seqlen/balanced_max:28754 - global_seqlen/mean:23272.0 - actor/entropy:0.09004729986190796 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6166199445724487 - training/rollout_probs_diff_mean:0.0027836794033646584 - training/rollout_probs_diff_std:0.011670615524053574 - training/rollout_actor_probs_pearson_corr:0.9972294569015503 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999050498008728 - rollout_corr/rollout_is_ratio_fraction_high:1.6847496226546355e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.423747931374237e-05 - rollout_corr/rollout_is_max:2.463893175125122 - rollout_corr/rollout_is_min:0.2046324759721756 - rollout_corr/rollout_is_std:0.03296531364321709 - rollout_corr/rollout_is_eff_sample_size:0.9989144677772216 - rollout_corr/rollout_is_seq_mean:0.9998564124107361 - rollout_corr/rollout_is_seq_std:0.0022072712890803814 - rollout_corr/rollout_is_seq_max:1.006649136543274 - rollout_corr/rollout_is_seq_min:0.99166339635849 - rollout_corr/rollout_is_seq_max_deviation:0.00833660364151001 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1013299119658768) - rollout_corr/training_log_ppl:np.float64(0.09491147637072572) - rollout_corr/kl:np.float64(0.0012193648983345184) - rollout_corr/k3_kl:np.float64(0.0009733567474512483) - rollout_corr/rollout_ppl:np.float64(1.099963684566319) - rollout_corr/rollout_log_ppl:np.float64(0.09369211093780905) - rollout_corr/log_ppl_diff:np.float64(0.0012193654329166748) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020960377587471157) - rollout_corr/log_ppl_diff_max:np.float64(0.05848722159862518) - rollout_corr/log_ppl_diff_min:np.float64(-0.007673636078834534) - rollout_corr/ppl_ratio:np.float64(1.001229693647474) - rollout_corr/chi2_token:np.float64(0.0014654879923909903) - rollout_corr/chi2_seq:np.float64(0.6356173786334693) - actor/pg_loss:np.float64(0.003317820141091943) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006650509092196444) - actor/ppo_kl:np.float64(0.0003957474698959018) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(62520.58305266849) - self_distillation/token_reweight_w_std:np.float64(1281642.6764327008) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001606633886695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03943118227471132) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.3942992687225342) - perf/mfu/actor:np.float64(0.039518949200975054) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.29967880249023) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0459052175283432 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0459052175283432 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:463.71875 - response_length/max:8192.0 - response_length/min:80.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:463.71875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.063258767127991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0605478826910257) - timing_s/agent_loop/generate_sequences/max:np.float64(49.57040260732174) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.396626845227729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.57040260732174) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.50206731632352 - timing_s/reward:0.05697246454656124 - timing_s/old_log_prob:2.633130395784974 - timing_s/adv:0.6030823420733213 - timing_s/update_actor:17.1904456615448 - timing_s/step:76.02233720384538 - timing_s/stop_profile:0.00010943599045276642 - timing_per_token_ms/update_actor:0.09233438070183482 - timing_per_token_ms/adv:0.003239313026777465 - timing_per_token_ms/gen:0.46753544137343755 - perf/total_num_tokens:186176 - perf/time_per_step:76.02233720384538 - perf/throughput:306.1205542470858 (TaskRunner pid=2074004) Training Progress: 29%|██▉ | 29/100 [24:03<1:01:22, 51.87s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We are given: (TaskRunner pid=2074004) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2074004) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2074004) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We need to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is **B: 56 pC**. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2074004) step:30 - global_seqlen/min:16055 - global_seqlen/max:23382 - global_seqlen/minmax_diff:7327 - global_seqlen/balanced_min:20479 - global_seqlen/balanced_max:20784 - global_seqlen/mean:20529.125 - actor/entropy:0.11640558391809464 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8577249050140381 - training/rollout_probs_diff_mean:0.0033130382653325796 - training/rollout_probs_diff_std:0.013030188158154488 - training/rollout_actor_probs_pearson_corr:0.9971984624862671 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0003026723861694 - rollout_corr/rollout_is_ratio_fraction_high:6.133902934379876e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010223171557299793 - rollout_corr/rollout_is_max:9.797200202941895 - rollout_corr/rollout_is_min:0.02608746662735939 - rollout_corr/rollout_is_std:0.03469718620181084 - rollout_corr/rollout_is_eff_sample_size:0.9987981476185248 - rollout_corr/rollout_is_seq_mean:1.0002856254577637 - rollout_corr/rollout_is_seq_std:0.0022839626763015985 - rollout_corr/rollout_is_seq_max:1.0106697082519531 - rollout_corr/rollout_is_seq_min:0.992483377456665 - rollout_corr/rollout_is_seq_max_deviation:0.010669708251953125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116787108592689) - rollout_corr/training_log_ppl:np.float64(0.10795232591772219) - rollout_corr/kl:np.float64(0.000616683089219805) - rollout_corr/k3_kl:np.float64(0.0007727639311809753) - rollout_corr/rollout_ppl:np.float64(1.1160784345120192) - rollout_corr/rollout_log_ppl:np.float64(0.10733564237307291) - rollout_corr/log_ppl_diff:np.float64(0.0006166835446492769) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018158721577492543) - rollout_corr/log_ppl_diff_max:np.float64(0.01455879956483841) - rollout_corr/log_ppl_diff_min:np.float64(-0.007083818316459656) - rollout_corr/ppl_ratio:np.float64(1.000619996106252) - rollout_corr/chi2_token:np.float64(0.0019336091354489326) - rollout_corr/chi2_seq:np.float64(1.0694307961966842) - actor/pg_loss:np.float64(0.002601322717964649) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044001030278195685) - actor/ppo_kl:np.float64(0.00017719261101882544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(32765.776810475392) - self_distillation/token_reweight_w_std:np.float64(708157.8602350631) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986124283168465) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04477454177686013) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.21477733738720417) - perf/mfu/actor:np.float64(0.03686629124421728) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.2654571533203) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10428943394865786) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8249500000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06501528454070343) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6504875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07057889100574483) - val-aux/sciknoweval/score/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/score/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10428943394865786) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8249500000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06501528454070343) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6504875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07057889100574483) - val-core/sciknoweval/acc/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10428943394865786) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8249500000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06501528454070343) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6504875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07057889100574483) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02339681404319957) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9992374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.007777136517816352) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9865124999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03086320051797221) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9931000000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.022986682202438165) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.97585) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.038177039681755176) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9976375000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.013409115755367873) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9586375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04250866307068464) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005306554090650112) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.93815) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03807095702890277) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.000683625445693766) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00048432277981191874 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00048432277981191874 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:382.09765625 - response_length/max:1653.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.09765625 - response_length_non_aborted/max:1653.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011514127254486084 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9901416879147291) - timing_s/agent_loop/generate_sequences/max:np.float64(10.789292139932513) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9146284311136696) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.789292139932513) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1653 - timing_s/gen:16.85866062529385 - timing_s/reward:0.059875741600990295 - timing_s/old_log_prob:2.412396924570203 - timing_s/adv:0.6014310400933027 - timing_s/update_actor:15.988812597468495 - timing_s/step:36.00774179585278 - timing_s/testing:91.78712977468967 - timing_s/save_checkpoint:6.680276766419411 - timing_s/stop_profile:0.00010959990322589874 - timing_per_token_ms/update_actor:0.09735444519352685 - timing_per_token_ms/adv:0.003662059635355274 - timing_per_token_ms/gen:0.17234898458646095 - perf/total_num_tokens:164233 - perf/time_per_step:36.00774179585278 - perf/throughput:570.1308656452446 (TaskRunner pid=2074004) Training Progress: 30%|███ | 30/100 [26:18<1:29:26, 76.67s/it] (TaskRunner pid=2074004) step:31 - global_seqlen/min:18767 - global_seqlen/max:37361 - global_seqlen/minmax_diff:18594 - global_seqlen/balanced_min:23793 - global_seqlen/balanced_max:23854 - global_seqlen/mean:23813.5 - actor/entropy:0.09906209260225296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35506775975227356 - training/rollout_probs_diff_mean:0.0027171920519322157 - training/rollout_probs_diff_std:0.01121894083917141 - training/rollout_actor_probs_pearson_corr:0.997601330280304 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0000742673873901 - rollout_corr/rollout_is_ratio_fraction_high:2.5133205781457946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.188867751508951e-05 - rollout_corr/rollout_is_max:2.3356175422668457 - rollout_corr/rollout_is_min:0.32783398032188416 - rollout_corr/rollout_is_std:0.030455991625785828 - rollout_corr/rollout_is_eff_sample_size:0.9990735300790685 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.0015059742145240307 - rollout_corr/rollout_is_seq_max:1.0055322647094727 - rollout_corr/rollout_is_seq_min:0.9959016442298889 - rollout_corr/rollout_is_seq_max_deviation:0.005532264709472656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.09525554953143) - rollout_corr/training_log_ppl:np.float64(0.08918771221215138) - rollout_corr/kl:np.float64(0.0004833923444760302) - rollout_corr/k3_kl:np.float64(0.0005884723202669306) - rollout_corr/rollout_ppl:np.float64(1.094714654609561) - rollout_corr/rollout_log_ppl:np.float64(0.08870431894683861) - rollout_corr/log_ppl_diff:np.float64(0.0004833932653127704) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014353545957419556) - rollout_corr/log_ppl_diff_max:np.float64(0.007750939577817917) - rollout_corr/log_ppl_diff_min:np.float64(-0.0094594806432724) - rollout_corr/ppl_ratio:np.float64(1.0004852761048824) - rollout_corr/chi2_token:np.float64(0.0013451201375573874) - rollout_corr/chi2_seq:np.float64(0.6003347004298121) - actor/pg_loss:np.float64(0.0021815065992996097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000496987612450539) - actor/ppo_kl:np.float64(0.0001865752313179314) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(91561.24236924388) - self_distillation/token_reweight_w_std:np.float64(1730255.973181627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005300368648022) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04233101321733557) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.2640515901148319) - perf/mfu/actor:np.float64(0.04318160738783638) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.296875) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013344685547053814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013344685547053814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.265625 - response_length/max:1752.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.265625 - response_length_non_aborted/max:1752.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.933486580848694e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.500905817374587) - timing_s/agent_loop/generate_sequences/max:np.float64(11.73058369755745) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.724863474373706) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.73058369755745) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1752 - timing_s/gen:17.499854369089007 - timing_s/reward:0.06082160025835037 - timing_s/old_log_prob:2.452167185023427 - timing_s/adv:0.642735943198204 - timing_s/update_actor:15.802205985412002 - timing_s/step:36.550422716885805 - timing_s/stop_profile:3.2784417271614075e-05 - timing_per_token_ms/update_actor:0.08294772915264452 - timing_per_token_ms/adv:0.003373800277144288 - timing_per_token_ms/gen:0.14660914822801688 - perf/total_num_tokens:190508 - perf/time_per_step:36.550422716885805 - perf/throughput:651.5246125730437 (TaskRunner pid=2074004) Training Progress: 31%|███ | 31/100 [26:54<1:14:20, 64.64s/it] (TaskRunner pid=2074004) step:32 - global_seqlen/min:14634 - global_seqlen/max:23574 - global_seqlen/minmax_diff:8940 - global_seqlen/balanced_min:18859 - global_seqlen/balanced_max:18877 - global_seqlen/mean:18872.25 - actor/entropy:0.10486175119876862 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43836408853530884 - training/rollout_probs_diff_mean:0.003058877307921648 - training/rollout_probs_diff_std:0.012399275787174702 - training/rollout_actor_probs_pearson_corr:0.9971876740455627 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999085068702698 - rollout_corr/rollout_is_ratio_fraction_high:2.3329055693466216e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001399743341607973 - rollout_corr/rollout_is_max:2.0753023624420166 - rollout_corr/rollout_is_min:0.2297990471124649 - rollout_corr/rollout_is_std:0.032623618841171265 - rollout_corr/rollout_is_eff_sample_size:0.998936593082401 - rollout_corr/rollout_is_seq_mean:0.9999740123748779 - rollout_corr/rollout_is_seq_std:0.0029045569244772196 - rollout_corr/rollout_is_seq_max:1.014037847518921 - rollout_corr/rollout_is_seq_min:0.9840235710144043 - rollout_corr/rollout_is_seq_max_deviation:0.015976428985595703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0999524341896176) - rollout_corr/training_log_ppl:np.float64(0.09279653291014256) - rollout_corr/kl:np.float64(0.0007732100730457603) - rollout_corr/k3_kl:np.float64(0.000771619481270136) - rollout_corr/rollout_ppl:np.float64(1.0990605875849724) - rollout_corr/rollout_log_ppl:np.float64(0.09202332204586128) - rollout_corr/log_ppl_diff:np.float64(0.0007732108642812818) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020941188995493576) - rollout_corr/log_ppl_diff_max:np.float64(0.02633790671825409) - rollout_corr/log_ppl_diff_min:np.float64(-0.011187225580215454) - rollout_corr/ppl_ratio:np.float64(1.0007794103585184) - rollout_corr/chi2_token:np.float64(0.0014043895062059164) - rollout_corr/chi2_seq:np.float64(0.37330453749746084) - actor/pg_loss:np.float64(0.0019935755990445614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005535308105208969) - actor/ppo_kl:np.float64(0.00010920122786206576) - actor/pg_clipfrac_lower:np.float64(7.207103408291005e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(222890.26285817288) - self_distillation/token_reweight_w_std:np.float64(2919852.600875127) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000309090828523) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03591290376789402) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3092605173587799) - perf/mfu/actor:np.float64(0.03432852611920223) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.16852188110352) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020937828812748194 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020937828812748194 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.8828125 - response_length/max:1186.0 - response_length/min:51.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.8828125 - response_length_non_aborted/max:1186.0 - response_length_non_aborted/min:51.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.379349648952484e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.595955204218626) - timing_s/agent_loop/generate_sequences/max:np.float64(8.028898818418384) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3284817362364265) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.028898818418384) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:1142 - timing_s/gen:13.935343297198415 - timing_s/reward:0.058442361652851105 - timing_s/old_log_prob:2.4146821219474077 - timing_s/adv:0.5538248158991337 - timing_s/update_actor:15.78230369463563 - timing_s/step:32.8320482596755 - timing_s/stop_profile:5.641952157020569e-05 - timing_per_token_ms/update_actor:0.10453379760386036 - timing_per_token_ms/adv:0.0036682484593724496 - timing_per_token_ms/gen:0.16254920444649965 - perf/total_num_tokens:150978 - perf/time_per_step:32.8320482596755 - perf/throughput:574.811837834041 (TaskRunner pid=2074004) Training Progress: 32%|███▏ | 32/100 [27:27<1:02:27, 55.11s/it] (TaskRunner pid=2074004) step:33 - global_seqlen/min:19167 - global_seqlen/max:32062 - global_seqlen/minmax_diff:12895 - global_seqlen/balanced_min:24910 - global_seqlen/balanced_max:31094 - global_seqlen/mean:25699.125 - actor/entropy:0.0963616594672203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5835636854171753 - training/rollout_probs_diff_mean:0.002623476553708315 - training/rollout_probs_diff_std:0.011050577275454998 - training/rollout_actor_probs_pearson_corr:0.9976565837860107 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000323057174683 - rollout_corr/rollout_is_ratio_fraction_high:2.285731716256123e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.571463432512246e-05 - rollout_corr/rollout_is_max:3.547874927520752 - rollout_corr/rollout_is_min:0.27111193537712097 - rollout_corr/rollout_is_std:0.030399184674024582 - rollout_corr/rollout_is_eff_sample_size:0.9990767427790138 - rollout_corr/rollout_is_seq_mean:1.0000334978103638 - rollout_corr/rollout_is_seq_std:0.0019103900995105505 - rollout_corr/rollout_is_seq_max:1.0073792934417725 - rollout_corr/rollout_is_seq_min:0.9880338907241821 - rollout_corr/rollout_is_seq_max_deviation:0.011966109275817871 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1100693135522306) - rollout_corr/training_log_ppl:np.float64(0.10072795890664565) - rollout_corr/kl:np.float64(0.0004943630928924136) - rollout_corr/k3_kl:np.float64(0.0005992158173953044) - rollout_corr/rollout_ppl:np.float64(1.1094494266435504) - rollout_corr/rollout_log_ppl:np.float64(0.10023359479600913) - rollout_corr/log_ppl_diff:np.float64(0.0004943641106365249) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015578279126202688) - rollout_corr/log_ppl_diff_max:np.float64(0.022818326950073242) - rollout_corr/log_ppl_diff_min:np.float64(-0.005478024482727051) - rollout_corr/ppl_ratio:np.float64(1.0004977090284228) - rollout_corr/chi2_token:np.float64(0.001456292113289237) - rollout_corr/chi2_seq:np.float64(0.8751170034520328) - actor/pg_loss:np.float64(0.0023588838521391153) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005614726733256248) - actor/ppo_kl:np.float64(6.841486872133373e-06) - actor/pg_clipfrac_lower:np.float64(4.124868155486183e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(73720.56168391788) - self_distillation/token_reweight_w_std:np.float64(1263082.4041759328) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995736547280103) - self_distillation/token_reweight_w_clip_frac:np.float64(0.038523146067745984) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.31921055912971497) - perf/mfu/actor:np.float64(0.04383612915635905) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.20793533325195) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010148649103939533 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010148649103939533 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:512.69140625 - response_length/max:8192.0 - response_length/min:90.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:512.69140625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001917630434036255 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0906069297343493) - timing_s/agent_loop/generate_sequences/max:np.float64(50.3866535294801) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.971057376002136) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.3866535294801) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.46107683330774 - timing_s/reward:0.06383568607270718 - timing_s/old_log_prob:2.6003563329577446 - timing_s/adv:0.6247836276888847 - timing_s/update_actor:17.063110006973147 - timing_s/step:76.9013549759984 - timing_s/stop_profile:0.00010899640619754791 - timing_per_token_ms/update_actor:0.0829946058813926 - timing_per_token_ms/adv:0.003038934339636489 - timing_per_token_ms/gen:0.4301829105997588 - perf/total_num_tokens:205593 - perf/time_per_step:76.9013549759984 - perf/throughput:334.1829933688545 (TaskRunner pid=2074004) Training Progress: 33%|███▎ | 33/100 [28:44<1:08:51, 61.66s/it] (TaskRunner pid=2074004) step:34 - global_seqlen/min:17957 - global_seqlen/max:26538 - global_seqlen/minmax_diff:8581 - global_seqlen/balanced_min:21718 - global_seqlen/balanced_max:21733 - global_seqlen/mean:21729.125 - actor/entropy:0.09093478322029114 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351693630218506 - training/rollout_probs_diff_mean:0.002648385940119624 - training/rollout_probs_diff_std:0.01128794439136982 - training/rollout_actor_probs_pearson_corr:0.9973158240318298 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001046657562256 - rollout_corr/rollout_is_ratio_fraction_high:3.717990330187604e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.29497618926689e-05 - rollout_corr/rollout_is_max:3.9687960147857666 - rollout_corr/rollout_is_min:0.3704526722431183 - rollout_corr/rollout_is_std:0.03138892725110054 - rollout_corr/rollout_is_eff_sample_size:0.9990158239877898 - rollout_corr/rollout_is_seq_mean:0.9999611377716064 - rollout_corr/rollout_is_seq_std:0.002279487904161215 - rollout_corr/rollout_is_seq_max:1.009277582168579 - rollout_corr/rollout_is_seq_min:0.9918825626373291 - rollout_corr/rollout_is_seq_max_deviation:0.009277582168579102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0867361915297806) - rollout_corr/training_log_ppl:np.float64(0.08140930149238557) - rollout_corr/kl:np.float64(0.0006834588668453989) - rollout_corr/k3_kl:np.float64(0.0006041397766978207) - rollout_corr/rollout_ppl:np.float64(1.0859681307338178) - rollout_corr/rollout_log_ppl:np.float64(0.08072584179171827) - rollout_corr/log_ppl_diff:np.float64(0.0006834597006672993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016792384558357298) - rollout_corr/log_ppl_diff_max:np.float64(0.011221528053283691) - rollout_corr/log_ppl_diff_min:np.float64(-0.004778027534484863) - rollout_corr/ppl_ratio:np.float64(1.000686518382281) - rollout_corr/chi2_token:np.float64(0.0010883505456149578) - rollout_corr/chi2_seq:np.float64(0.5784096252173185) - actor/pg_loss:np.float64(0.002307310584001243) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005154957439117425) - actor/ppo_kl:np.float64(9.805531538464862e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(82543.4987949559) - self_distillation/token_reweight_w_std:np.float64(1531421.5975041706) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002939573023468) - self_distillation/token_reweight_w_clip_frac:np.float64(0.032446051089209504) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.2603403516113758) - perf/mfu/actor:np.float64(0.0392491945722704) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.23517227172852) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008258585818111897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008258585818111897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.25390625 - response_length/max:1275.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.25390625 - response_length_non_aborted/max:1275.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014248304069042206 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9343549367040396) - timing_s/agent_loop/generate_sequences/max:np.float64(9.114851927384734) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.215552492758434) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.114851927384734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1275 - timing_s/gen:14.953994173556566 - timing_s/reward:0.05961340852081776 - timing_s/old_log_prob:2.531619004905224 - timing_s/adv:0.5816438850015402 - timing_s/update_actor:15.924356224015355 - timing_s/step:34.133271373808384 - timing_s/stop_profile:4.3351203203201294e-05 - timing_per_token_ms/update_actor:0.09160721050672402 - timing_per_token_ms/adv:0.003345992331729535 - timing_per_token_ms/gen:0.13899701792588712 - perf/total_num_tokens:173833 - perf/time_per_step:34.133271373808384 - perf/throughput:636.5966145475729 (TaskRunner pid=2074004) Training Progress: 34%|███▍ | 34/100 [29:18<58:45, 53.41s/it] (TaskRunner pid=2074004) step:35 - global_seqlen/min:17441 - global_seqlen/max:33644 - global_seqlen/minmax_diff:16203 - global_seqlen/balanced_min:21811 - global_seqlen/balanced_max:28467 - global_seqlen/mean:22649.75 - actor/entropy:0.10204526782035828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.293226420879364 - training/rollout_probs_diff_mean:0.002843219321221113 - training/rollout_probs_diff_std:0.011386570520699024 - training/rollout_actor_probs_pearson_corr:0.9975972771644592 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.000219702720642 - rollout_corr/rollout_is_ratio_fraction_high:1.7400079741491936e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011310051922919229 - rollout_corr/rollout_is_max:2.0903584957122803 - rollout_corr/rollout_is_min:0.10869106650352478 - rollout_corr/rollout_is_std:0.03217843919992447 - rollout_corr/rollout_is_eff_sample_size:0.9989663329398729 - rollout_corr/rollout_is_seq_mean:1.0003238916397095 - rollout_corr/rollout_is_seq_std:0.002396445255726576 - rollout_corr/rollout_is_seq_max:1.0162620544433594 - rollout_corr/rollout_is_seq_min:0.9926619529724121 - rollout_corr/rollout_is_seq_max_deviation:0.016262054443359375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1133964317850769) - rollout_corr/training_log_ppl:np.float64(0.10447760274473694) - rollout_corr/kl:np.float64(0.000506502414840071) - rollout_corr/k3_kl:np.float64(0.0008595675468114905) - rollout_corr/rollout_ppl:np.float64(1.1128244805149734) - rollout_corr/rollout_log_ppl:np.float64(0.10397110023404821) - rollout_corr/log_ppl_diff:np.float64(0.0005065025106887333) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018531291279941797) - rollout_corr/log_ppl_diff_max:np.float64(0.026727454736828804) - rollout_corr/log_ppl_diff_min:np.float64(-0.01405394822359085) - rollout_corr/ppl_ratio:np.float64(1.000511220889166) - rollout_corr/chi2_token:np.float64(0.0022407646756619215) - rollout_corr/chi2_seq:np.float64(1.0442664730362594) - actor/pg_loss:np.float64(0.002358463010750711) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048152222007047385) - actor/ppo_kl:np.float64(0.0001821644597370664) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(110187.43466792861) - self_distillation/token_reweight_w_std:np.float64(1515968.7875927906) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997799128759652) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034502132126362994) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.34709084033966064) - perf/mfu/actor:np.float64(0.0391226831248303) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.17388153076172) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.053757548332214355 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.053757548332214355 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:448.9921875 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:448.9921875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.4147873520851135e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8165026474744081) - timing_s/agent_loop/generate_sequences/max:np.float64(50.60485251620412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.408925204807019) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.60485251620412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.32023567520082 - timing_s/reward:0.0652729794383049 - timing_s/old_log_prob:2.546900112181902 - timing_s/adv:0.5270522274076939 - timing_s/update_actor:16.483777582645416 - timing_s/step:76.03851772285998 - timing_s/stop_profile:0.00011594034731388092 - timing_per_token_ms/update_actor:0.0909710790552071 - timing_per_token_ms/adv:0.0029087088566523574 - timing_per_token_ms/gen:0.489988304320447 - perf/total_num_tokens:181198 - perf/time_per_step:76.03851772285998 - perf/throughput:297.87206113817564 (TaskRunner pid=2074004) Training Progress: 35%|███▌ | 35/100 [30:34<1:05:13, 60.21s/it] (TaskRunner pid=2074004) step:36 - global_seqlen/min:18954 - global_seqlen/max:33911 - global_seqlen/minmax_diff:14957 - global_seqlen/balanced_min:26378 - global_seqlen/balanced_max:32662 - global_seqlen/mean:27187.75 - actor/entropy:0.1043354868888855 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4853993356227875 - training/rollout_probs_diff_mean:0.0027279683854430914 - training/rollout_probs_diff_std:0.011042559519410133 - training/rollout_actor_probs_pearson_corr:0.9978550672531128 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.000059723854065 - rollout_corr/rollout_is_ratio_fraction_high:2.6941470423480496e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.082440763246268e-05 - rollout_corr/rollout_is_max:7.419532775878906 - rollout_corr/rollout_is_min:0.13505788147449493 - rollout_corr/rollout_is_std:0.03132620081305504 - rollout_corr/rollout_is_eff_sample_size:0.9990196311945923 - rollout_corr/rollout_is_seq_mean:1.000086784362793 - rollout_corr/rollout_is_seq_std:0.0020494619384407997 - rollout_corr/rollout_is_seq_max:1.015174388885498 - rollout_corr/rollout_is_seq_min:0.99062579870224 - rollout_corr/rollout_is_seq_max_deviation:0.015174388885498047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1137358797714114) - rollout_corr/training_log_ppl:np.float64(0.10603692273616616) - rollout_corr/kl:np.float64(0.0006563753371509051) - rollout_corr/k3_kl:np.float64(0.0007457188598607445) - rollout_corr/rollout_ppl:np.float64(1.1129927351139486) - rollout_corr/rollout_log_ppl:np.float64(0.10538054626522353) - rollout_corr/log_ppl_diff:np.float64(0.0006563764709426323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015180988721112953) - rollout_corr/log_ppl_diff_max:np.float64(0.011238820850849152) - rollout_corr/log_ppl_diff_min:np.float64(-0.007395319640636444) - rollout_corr/ppl_ratio:np.float64(1.0006589936092496) - rollout_corr/chi2_token:np.float64(0.002004736103117466) - rollout_corr/chi2_seq:np.float64(1.2244051673915237) - actor/pg_loss:np.float64(0.0032588703325018287) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008896479857867234) - actor/ppo_kl:np.float64(8.400913770501006e-05) - actor/pg_clipfrac_lower:np.float64(5.440459517558338e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(142463.33054139256) - self_distillation/token_reweight_w_std:np.float64(2692305.3041795073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015312230680138) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0585212862060871) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.31766650825738907) - perf/mfu/actor:np.float64(0.04633240200426601) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.1927490234375) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04938371479511261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04938371479511261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:579.9609375 - response_length/max:8192.0 - response_length/min:82.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:579.9609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.186511695384979e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9956839792430401) - timing_s/agent_loop/generate_sequences/max:np.float64(51.712549494579434) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.89511316808057) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.712549494579434) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.78715647570789 - timing_s/reward:0.06627766788005829 - timing_s/old_log_prob:2.5614236313849688 - timing_s/adv:0.6225856561213732 - timing_s/update_actor:17.082536520436406 - timing_s/step:78.20577510073781 - timing_s/stop_profile:0.00011078454554080963 - timing_per_token_ms/update_actor:0.07853967559119643 - timing_per_token_ms/adv:0.0028624364655100787 - timing_per_token_ms/gen:0.3892177306911018 - perf/total_num_tokens:217502 - perf/time_per_step:78.20577510073781 - perf/throughput:347.6437637115562 (TaskRunner pid=2074004) Training Progress: 36%|███▌ | 36/100 [31:52<1:09:59, 65.62s/it] (TaskRunner pid=2074004) step:37 - global_seqlen/min:18791 - global_seqlen/max:27840 - global_seqlen/minmax_diff:9049 - global_seqlen/balanced_min:23579 - global_seqlen/balanced_max:23755 - global_seqlen/mean:23627.625 - actor/entropy:0.10102436691522598 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39139458537101746 - training/rollout_probs_diff_mean:0.002805094001814723 - training/rollout_probs_diff_std:0.011440390720963478 - training/rollout_actor_probs_pearson_corr:0.9976106882095337 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9998602271080017 - rollout_corr/rollout_is_ratio_fraction_high:1.6350957594113424e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.9052869144361466e-05 - rollout_corr/rollout_is_max:2.243018388748169 - rollout_corr/rollout_is_min:0.0117238974198699 - rollout_corr/rollout_is_std:0.03203921392560005 - rollout_corr/rollout_is_eff_sample_size:0.9989743035225582 - rollout_corr/rollout_is_seq_mean:0.9999796152114868 - rollout_corr/rollout_is_seq_std:0.002273126505315304 - rollout_corr/rollout_is_seq_max:1.0130821466445923 - rollout_corr/rollout_is_seq_min:0.9926750659942627 - rollout_corr/rollout_is_seq_max_deviation:0.013082146644592285 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0996112595312297) - rollout_corr/training_log_ppl:np.float64(0.09333795910788467) - rollout_corr/kl:np.float64(0.001013359834161065) - rollout_corr/k3_kl:np.float64(0.0008460905967808685) - rollout_corr/rollout_ppl:np.float64(1.0984738464467227) - rollout_corr/rollout_log_ppl:np.float64(0.09232459875784116) - rollout_corr/log_ppl_diff:np.float64(0.0010133603500435129) - rollout_corr/log_ppl_abs_diff:np.float64(0.00191976049245568) - rollout_corr/log_ppl_diff_max:np.float64(0.014345422387123108) - rollout_corr/log_ppl_diff_min:np.float64(-0.009604066610336304) - rollout_corr/ppl_ratio:np.float64(1.001017474103719) - rollout_corr/chi2_token:np.float64(0.001261905301362276) - rollout_corr/chi2_seq:np.float64(0.5114637641236186) - actor/pg_loss:np.float64(0.003147437470033765) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007599365783335088) - actor/ppo_kl:np.float64(0.0003716098390409428) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(192101.5823432547) - self_distillation/token_reweight_w_std:np.float64(2797782.555044028) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008915911894292) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04522743531560991) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.35525741800665855) - perf/mfu/actor:np.float64(0.04261905670880045) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.2181911468506) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008520892821252346 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008520892821252346 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.80078125 - response_length/max:2318.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.80078125 - response_length_non_aborted/max:2318.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.429400622844696e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.102793700993061) - timing_s/agent_loop/generate_sequences/max:np.float64(14.33990266919136) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5264077913961955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.33990266919136) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2318 - timing_s/gen:20.240060921758413 - timing_s/reward:0.06487942114472389 - timing_s/old_log_prob:2.515492904931307 - timing_s/adv:0.5939651485532522 - timing_s/update_actor:16.080622201785445 - timing_s/step:39.57463042251766 - timing_s/stop_profile:0.00011385232210159302 - timing_per_token_ms/update_actor:0.08507320457401794 - timing_per_token_ms/adv:0.0031423235966017123 - timing_per_token_ms/gen:0.16547218229484384 - perf/total_num_tokens:189021 - perf/time_per_step:39.57463042251766 - perf/throughput:597.0396879955716 (TaskRunner pid=2074004) Training Progress: 37%|███▋ | 37/100 [32:32<1:00:42, 57.82s/it] (TaskRunner pid=2074004) step:38 - global_seqlen/min:17600 - global_seqlen/max:27502 - global_seqlen/minmax_diff:9902 - global_seqlen/balanced_min:21161 - global_seqlen/balanced_max:21168 - global_seqlen/mean:21164.5 - actor/entropy:0.09618508070707321 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138811707496643 - training/rollout_probs_diff_mean:0.0026226078625768423 - training/rollout_probs_diff_std:0.011445230804383755 - training/rollout_actor_probs_pearson_corr:0.9974937438964844 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000364780426025 - rollout_corr/rollout_is_ratio_fraction_high:1.979805892915465e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.899029828375205e-05 - rollout_corr/rollout_is_max:2.0751965045928955 - rollout_corr/rollout_is_min:0.013780022040009499 - rollout_corr/rollout_is_std:0.03070547617971897 - rollout_corr/rollout_is_eff_sample_size:0.9990580618130113 - rollout_corr/rollout_is_seq_mean:1.0000827312469482 - rollout_corr/rollout_is_seq_std:0.0019129818538203835 - rollout_corr/rollout_is_seq_max:1.0122212171554565 - rollout_corr/rollout_is_seq_min:0.9918540716171265 - rollout_corr/rollout_is_seq_max_deviation:0.012221217155456543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0946312695741653) - rollout_corr/training_log_ppl:np.float64(0.08725073869572952) - rollout_corr/kl:np.float64(0.0006405033833267737) - rollout_corr/k3_kl:np.float64(0.0005329421040762838) - rollout_corr/rollout_ppl:np.float64(1.0939017231576145) - rollout_corr/rollout_log_ppl:np.float64(0.08661023495005793) - rollout_corr/log_ppl_diff:np.float64(0.0006405037456715945) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014858933245704975) - rollout_corr/log_ppl_diff_max:np.float64(0.014180347323417664) - rollout_corr/log_ppl_diff_min:np.float64(-0.004229940474033356) - rollout_corr/ppl_ratio:np.float64(1.0006429618224502) - rollout_corr/chi2_token:np.float64(0.0008146455511450768) - rollout_corr/chi2_seq:np.float64(0.17395253549329937) - actor/pg_loss:np.float64(0.002348349546082318) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007172689474828076) - actor/ppo_kl:np.float64(0.00023065414274014984) - actor/pg_clipfrac_lower:np.float64(1.249999968422344e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(223555.59363828856) - self_distillation/token_reweight_w_std:np.float64(3377666.5284842458) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001167273381725) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04519080827594735) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2977074235677719) - perf/mfu/actor:np.float64(0.03844596025288242) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.3445816040039) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034980697091668844 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034980697091668844 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.609375 - response_length/max:1200.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.609375 - response_length_non_aborted/max:1200.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.56140798330307e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0009879916906357) - timing_s/agent_loop/generate_sequences/max:np.float64(8.732898579910398) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.088401710039761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.732898579910398) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1200 - timing_s/gen:14.707298658788204 - timing_s/reward:0.05728561617434025 - timing_s/old_log_prob:2.3959624897688627 - timing_s/adv:0.5882252883166075 - timing_s/update_actor:15.931603338569403 - timing_s/step:33.75794658437371 - timing_s/stop_profile:0.0001205597072839737 - timing_per_token_ms/update_actor:0.09409390334386238 - timing_per_token_ms/adv:0.0034741270069964295 - timing_per_token_ms/gen:0.1455879890990715 - perf/total_num_tokens:169316 - perf/time_per_step:33.75794658437371 - perf/throughput:626.9486785015793 (TaskRunner pid=2074004) Training Progress: 38%|███▊ | 38/100 [33:06<52:18, 50.62s/it] (TaskRunner pid=2074004) step:39 - global_seqlen/min:14192 - global_seqlen/max:21395 - global_seqlen/minmax_diff:7203 - global_seqlen/balanced_min:18859 - global_seqlen/balanced_max:18868 - global_seqlen/mean:18864.75 - actor/entropy:0.08581894636154175 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8518780469894409 - training/rollout_probs_diff_mean:0.0027508889324963093 - training/rollout_probs_diff_std:0.012479580007493496 - training/rollout_actor_probs_pearson_corr:0.9966431260108948 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.82421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999743700027466 - rollout_corr/rollout_is_ratio_fraction_high:2.358101301069837e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010611455945763737 - rollout_corr/rollout_is_max:2.9851624965667725 - rollout_corr/rollout_is_min:0.14805817604064941 - rollout_corr/rollout_is_std:0.03196844086050987 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0000545978546143 - rollout_corr/rollout_is_seq_std:0.0020085289143025875 - rollout_corr/rollout_is_seq_max:1.0090090036392212 - rollout_corr/rollout_is_seq_min:0.9939286708831787 - rollout_corr/rollout_is_seq_max_deviation:0.009009003639221191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.090072206221521) - rollout_corr/training_log_ppl:np.float64(0.08314310060450225) - rollout_corr/kl:np.float64(0.0007816835961591551) - rollout_corr/k3_kl:np.float64(0.0008022212134584095) - rollout_corr/rollout_ppl:np.float64(1.0891828034073114) - rollout_corr/rollout_log_ppl:np.float64(0.08236141606539604) - rollout_corr/log_ppl_diff:np.float64(0.0007816845391062088) - rollout_corr/log_ppl_abs_diff:np.float64(0.00177329323196318) - rollout_corr/log_ppl_diff_max:np.float64(0.045620523393154144) - rollout_corr/log_ppl_diff_min:np.float64(-0.008172988891601562) - rollout_corr/ppl_ratio:np.float64(1.0007883713115007) - rollout_corr/chi2_token:np.float64(0.0012875364627689123) - rollout_corr/chi2_seq:np.float64(0.3463961035013199) - actor/pg_loss:np.float64(0.0016197055811062455) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035407407904131105) - actor/ppo_kl:np.float64(0.00027624938138259836) - actor/pg_clipfrac_lower:np.float64(1.398360836901702e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_w_mean:np.float64(47769.31919008936) - self_distillation/token_reweight_w_std:np.float64(767346.1878132067) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993690950796008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02541799924802035) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.3479584529995918) - perf/mfu/actor:np.float64(0.0347302901051022) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.15287017822266) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007721307687461376 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007721307687461376 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:331.3046875 - response_length/max:879.0 - response_length/min:65.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.3046875 - response_length_non_aborted/max:879.0 - response_length_non_aborted/min:65.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011703930795192719 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9549711402505636) - timing_s/agent_loop/generate_sequences/max:np.float64(6.732800455763936) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4961698439146858) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.732800455763936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:879 - timing_s/gen:12.607269439846277 - timing_s/reward:0.05702884867787361 - timing_s/old_log_prob:2.344845812767744 - timing_s/adv:0.5173551607877016 - timing_s/update_actor:15.629758136346936 - timing_s/step:31.24427343159914 - timing_s/stop_profile:0.000109916552901268 - timing_per_token_ms/update_actor:0.10356457239260351 - timing_per_token_ms/adv:0.003428054710423552 - timing_per_token_ms/gen:0.14864608955887326 - perf/total_num_tokens:150918 - perf/time_per_step:31.24427343159914 - perf/throughput:603.7826432833924 (TaskRunner pid=2074004) Training Progress: 39%|███▉ | 39/100 [33:37<45:34, 44.83s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2074004) step:40 - global_seqlen/min:17338 - global_seqlen/max:29776 - global_seqlen/minmax_diff:12438 - global_seqlen/balanced_min:22374 - global_seqlen/balanced_max:22391 - global_seqlen/mean:22384.625 - actor/entropy:0.089930459856987 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3171451985836029 - training/rollout_probs_diff_mean:0.0025929557159543037 - training/rollout_probs_diff_std:0.011268679052591324 - training/rollout_actor_probs_pearson_corr:0.9974833726882935 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:1.000073790550232 - rollout_corr/rollout_is_ratio_fraction_high:1.7818165360949934e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.4545413402374834e-05 - rollout_corr/rollout_is_max:2.4635725021362305 - rollout_corr/rollout_is_min:0.2820615768432617 - rollout_corr/rollout_is_std:0.03070935793220997 - rollout_corr/rollout_is_eff_sample_size:0.9990580618130113 - rollout_corr/rollout_is_seq_mean:0.9999992847442627 - rollout_corr/rollout_is_seq_std:0.001931522972881794 - rollout_corr/rollout_is_seq_max:1.0066325664520264 - rollout_corr/rollout_is_seq_min:0.9905868768692017 - rollout_corr/rollout_is_seq_max_deviation:0.00941312313079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0896218931302428) - rollout_corr/training_log_ppl:np.float64(0.08464303463551914) - rollout_corr/kl:np.float64(0.0005157805179152319) - rollout_corr/k3_kl:np.float64(0.0005807906663477524) - rollout_corr/rollout_ppl:np.float64(1.0890426081605256) - rollout_corr/rollout_log_ppl:np.float64(0.08412725336529547) - rollout_corr/log_ppl_diff:np.float64(0.0005157812702236697) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015912332892185077) - rollout_corr/log_ppl_diff_max:np.float64(0.009794257581233978) - rollout_corr/log_ppl_diff_min:np.float64(-0.0059789493680000305) - rollout_corr/ppl_ratio:np.float64(1.0005182197783142) - rollout_corr/chi2_token:np.float64(0.0013064609374850988) - rollout_corr/chi2_seq:np.float64(1.130385466851294) - actor/pg_loss:np.float64(0.002167836995795369) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005737871229030134) - actor/ppo_kl:np.float64(1.1175101938931675e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(140985.0619100898) - self_distillation/token_reweight_w_std:np.float64(2265311.5997434272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001305928453803) - self_distillation/token_reweight_w_clip_frac:np.float64(0.036739757866598666) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.2651498392224312) - perf/mfu/actor:np.float64(0.04031842311642975) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.09762573242188) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.625) - val-aux/sciknoweval/reward/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1088052134913493) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8369375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05406867605156187) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.648425) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08121353362515363) - val-aux/sciknoweval/score/mean@16:np.float64(0.625) - val-aux/sciknoweval/score/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1088052134913493) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8369375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05406867605156187) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.648425) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08121353362515363) - val-core/sciknoweval/acc/mean@16:np.float64(0.625) - val-aux/sciknoweval/acc/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1088052134913493) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8369375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05406867605156187) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.648425) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08121353362515363) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.024141732305909894) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989125000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.008527445661552757) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.985275) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03137280736233215) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9916750000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.024734393444018433) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9734375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03851008868178078) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9960000000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01673449238498939) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.956475) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04152269363307004) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9987) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00904727412664771) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.936625) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03653224445364453) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999775) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002781121632849242) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000915408250875771 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000915408250875771 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:438.45703125 - response_length/max:1467.0 - response_length/min:64.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:438.45703125 - response_length_non_aborted/max:1467.0 - response_length_non_aborted/min:64.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014247745275497437 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8024486973881721) - timing_s/agent_loop/generate_sequences/max:np.float64(10.08868633955717) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.353498277778272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.08868633955717) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1467 - timing_s/gen:16.001727301627398 - timing_s/reward:0.061644136905670166 - timing_s/old_log_prob:2.4333597384393215 - timing_s/adv:0.5891087763011456 - timing_s/update_actor:15.919175075367093 - timing_s/step:35.09279179573059 - timing_s/testing:93.67140794359148 - timing_s/save_checkpoint:6.656575288623571 - timing_s/stop_profile:0.00014066509902477264 - timing_per_token_ms/update_actor:0.08889569891927547 - timing_per_token_ms/adv:0.0032896953617781487 - timing_per_token_ms/gen:0.14256071363203168 - perf/total_num_tokens:179077 - perf/time_per_step:35.09279179573059 - perf/throughput:637.8695981299306 (TaskRunner pid=2074004) Training Progress: 40%|████ | 40/100 [35:53<1:12:01, 72.03s/it] (TaskRunner pid=2074004) step:41 - global_seqlen/min:20646 - global_seqlen/max:30800 - global_seqlen/minmax_diff:10154 - global_seqlen/balanced_min:23788 - global_seqlen/balanced_max:23817 - global_seqlen/mean:23809.0 - actor/entropy:0.08412408828735352 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083496570587158 - training/rollout_probs_diff_mean:0.002506777411326766 - training/rollout_probs_diff_std:0.010917266830801964 - training/rollout_actor_probs_pearson_corr:0.9974930882453918 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:1.6747051631682552e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.698820652673021e-05 - rollout_corr/rollout_is_max:2.611849784851074 - rollout_corr/rollout_is_min:0.2993384003639221 - rollout_corr/rollout_is_std:0.030081849545240402 - rollout_corr/rollout_is_eff_sample_size:0.999095900418749 - rollout_corr/rollout_is_seq_mean:0.9999955296516418 - rollout_corr/rollout_is_seq_std:0.00155645702034235 - rollout_corr/rollout_is_seq_max:1.0051690340042114 - rollout_corr/rollout_is_seq_min:0.9948093891143799 - rollout_corr/rollout_is_seq_max_deviation:0.005190610885620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.084649627096951) - rollout_corr/training_log_ppl:np.float64(0.08000787740093074) - rollout_corr/kl:np.float64(0.00042501650833115523) - rollout_corr/k3_kl:np.float64(0.0005248165592366405) - rollout_corr/rollout_ppl:np.float64(1.084169534035027) - rollout_corr/rollout_log_ppl:np.float64(0.07958286030770978) - rollout_corr/log_ppl_diff:np.float64(0.0004250170932209585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013505659480870236) - rollout_corr/log_ppl_diff_max:np.float64(0.006395198404788971) - rollout_corr/log_ppl_diff_min:np.float64(-0.0090675950050354) - rollout_corr/ppl_ratio:np.float64(1.0004267231561244) - rollout_corr/chi2_token:np.float64(0.0012936093844473362) - rollout_corr/chi2_seq:np.float64(1.8906177775934339) - actor/pg_loss:np.float64(0.002206359291449189) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006284501350819482) - actor/ppo_kl:np.float64(-4.702512273624393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(146167.46774941264) - self_distillation/token_reweight_w_std:np.float64(2535003.0120101087) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000315876677632) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04627777053974569) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.32163915038108826) - perf/mfu/actor:np.float64(0.04315536156757415) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.23479080200195) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009857733733952045 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009857733733952045 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.5 - response_length/max:1590.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.5 - response_length_non_aborted/max:1590.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.379800081253052e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2317185122519732) - timing_s/agent_loop/generate_sequences/max:np.float64(10.718674024567008) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.755297555726429) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.718674024567008) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1590 - timing_s/gen:16.80528495274484 - timing_s/reward:0.0609660129994154 - timing_s/old_log_prob:2.4625763557851315 - timing_s/adv:0.5879520736634731 - timing_s/update_actor:15.929620191454887 - timing_s/step:35.9443339202553 - timing_s/stop_profile:0.00010867416858673096 - timing_per_token_ms/update_actor:0.08363234591674833 - timing_per_token_ms/adv:0.003086816296691761 - timing_per_token_ms/gen:0.14071949484814478 - perf/total_num_tokens:190472 - perf/time_per_step:35.9443339202553 - perf/throughput:662.3853443166236 (TaskRunner pid=2074004) Training Progress: 41%|████ | 41/100 [36:29<1:00:11, 61.22s/it] (TaskRunner pid=2074004) step:42 - global_seqlen/min:20360 - global_seqlen/max:28679 - global_seqlen/minmax_diff:8319 - global_seqlen/balanced_min:24195 - global_seqlen/balanced_max:24221 - global_seqlen/mean:24215.75 - actor/entropy:0.09680385887622833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6651126146316528 - training/rollout_probs_diff_mean:0.00268166302703321 - training/rollout_probs_diff_std:0.011481569148600101 - training/rollout_actor_probs_pearson_corr:0.9974598288536072 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999688863754272 - rollout_corr/rollout_is_ratio_fraction_high:2.390247755101882e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001195123913930729 - rollout_corr/rollout_is_max:3.0732650756835938 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.030898984521627426 - rollout_corr/rollout_is_eff_sample_size:0.99904604449077 - rollout_corr/rollout_is_seq_mean:0.999945342540741 - rollout_corr/rollout_is_seq_std:0.001563522731885314 - rollout_corr/rollout_is_seq_max:1.0046355724334717 - rollout_corr/rollout_is_seq_min:0.9948148131370544 - rollout_corr/rollout_is_seq_max_deviation:0.005185186862945557 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0920318001881242) - rollout_corr/training_log_ppl:np.float64(0.0861618878916488) - rollout_corr/kl:np.float64(0.0007285167887651767) - rollout_corr/k3_kl:np.float64(0.0005457256108627462) - rollout_corr/rollout_ppl:np.float64(1.0912344371899962) - rollout_corr/rollout_log_ppl:np.float64(0.08543337112496374) - rollout_corr/log_ppl_diff:np.float64(0.0007285167666850612) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013972546003060415) - rollout_corr/log_ppl_diff_max:np.float64(0.014810048043727875) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045769959688186646) - rollout_corr/ppl_ratio:np.float64(1.0007309333886951) - rollout_corr/chi2_token:np.float64(0.0006357615347951651) - rollout_corr/chi2_seq:np.float64(0.37331055640242994) - actor/pg_loss:np.float64(0.002819624845869839) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005845594248512498) - actor/ppo_kl:np.float64(0.00019738566773241928) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(118144.65402136883) - self_distillation/token_reweight_w_std:np.float64(2489355.6096611773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007247163448483) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0432515628344845) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.2655489929020405) - perf/mfu/actor:np.float64(0.043623625719631715) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.27901077270508) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013405305799096823 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013405305799096823 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:490.2734375 - response_length/max:1590.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:490.2734375 - response_length_non_aborted/max:1590.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001036711037158966 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1733751446008682) - timing_s/agent_loop/generate_sequences/max:np.float64(11.362804541364312) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.0733919305566815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.362804541364312) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1590 - timing_s/gen:17.271388456225395 - timing_s/reward:0.06370862200856209 - timing_s/old_log_prob:2.4125118106603622 - timing_s/adv:0.5772267878055573 - timing_s/update_actor:16.07286419905722 - timing_s/step:36.48842146247625 - timing_s/stop_profile:0.00012125447392463684 - timing_per_token_ms/update_actor:0.08296699564878861 - timing_per_token_ms/adv:0.002979604120281001 - timing_per_token_ms/gen:0.13760966023604013 - perf/total_num_tokens:193726 - perf/time_per_step:36.48842146247625 - perf/throughput:663.6557304870765 (TaskRunner pid=2074004) Training Progress: 42%|████▏ | 42/100 [37:05<52:01, 53.81s/it] (TaskRunner pid=2074004) step:43 - global_seqlen/min:18165 - global_seqlen/max:29195 - global_seqlen/minmax_diff:11030 - global_seqlen/balanced_min:23675 - global_seqlen/balanced_max:23697 - global_seqlen/mean:23692.125 - actor/entropy:0.1023547351360321 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8032844066619873 - training/rollout_probs_diff_mean:0.0026671267114579678 - training/rollout_probs_diff_std:0.01125924102962017 - training/rollout_actor_probs_pearson_corr:0.9977118372917175 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.0000076293945312 - rollout_corr/rollout_is_ratio_fraction_high:1.6602332834736444e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012451749353203923 - rollout_corr/rollout_is_max:2.4638938903808594 - rollout_corr/rollout_is_min:0.01720614731311798 - rollout_corr/rollout_is_std:0.03037957102060318 - rollout_corr/rollout_is_eff_sample_size:0.9990778136835871 - rollout_corr/rollout_is_seq_mean:1.0000402927398682 - rollout_corr/rollout_is_seq_std:0.0017050451133400202 - rollout_corr/rollout_is_seq_max:1.0078315734863281 - rollout_corr/rollout_is_seq_min:0.9947986006736755 - rollout_corr/rollout_is_seq_max_deviation:0.007831573486328125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1006618356332183) - rollout_corr/training_log_ppl:np.float64(0.09350979735609144) - rollout_corr/kl:np.float64(0.000630945518594217) - rollout_corr/k3_kl:np.float64(0.0005851791151911812) - rollout_corr/rollout_ppl:np.float64(1.0999378017149866) - rollout_corr/rollout_log_ppl:np.float64(0.09287885023513809) - rollout_corr/log_ppl_diff:np.float64(0.0006309471209533513) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015661002835258842) - rollout_corr/log_ppl_diff_max:np.float64(0.007478758692741394) - rollout_corr/log_ppl_diff_min:np.float64(-0.007118046283721924) - rollout_corr/ppl_ratio:np.float64(1.000633135670796) - rollout_corr/chi2_token:np.float64(0.0010510697029531002) - rollout_corr/chi2_seq:np.float64(0.48250076407566667) - actor/pg_loss:np.float64(0.0034622400999069214) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007893792012509948) - actor/ppo_kl:np.float64(0.00017401028439456567) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(130333.681850672) - self_distillation/token_reweight_w_std:np.float64(2574542.6272122907) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005807324778289) - self_distillation/token_reweight_w_clip_frac:np.float64(0.056435555168718565) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.34318234771490097) - perf/mfu/actor:np.float64(0.04306781395749705) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.32390975952148) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010272692888975143 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010272692888975143 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.56640625 - response_length/max:1498.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.56640625 - response_length_non_aborted/max:1498.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.386613965034485e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1191709153354168) - timing_s/agent_loop/generate_sequences/max:np.float64(10.657457020133734) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.579246542722103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.657457020133734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1498 - timing_s/gen:16.503861967474222 - timing_s/reward:0.05490053817629814 - timing_s/old_log_prob:2.4478375520557165 - timing_s/adv:0.5797670539468527 - timing_s/update_actor:15.963536273688078 - timing_s/step:35.651620699092746 - timing_s/stop_profile:3.4825876355171204e-05 - timing_per_token_ms/update_actor:0.08422385219607822 - timing_per_token_ms/adv:0.003058859504723894 - timing_per_token_ms/gen:0.1370013030131094 - perf/total_num_tokens:189537 - perf/time_per_step:35.651620699092746 - perf/throughput:664.545525152042 (TaskRunner pid=2074004) Training Progress: 43%|████▎ | 43/100 [37:41<45:57, 48.37s/it] (TaskRunner pid=2074004) step:44 - global_seqlen/min:18372 - global_seqlen/max:30473 - global_seqlen/minmax_diff:12101 - global_seqlen/balanced_min:24351 - global_seqlen/balanced_max:24371 - global_seqlen/mean:24365.0 - actor/entropy:0.0982256531715393 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797517716884613 - training/rollout_probs_diff_mean:0.0026915573980659246 - training/rollout_probs_diff_std:0.011107339523732662 - training/rollout_actor_probs_pearson_corr:0.9976074695587158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0001195669174194 - rollout_corr/rollout_is_ratio_fraction_high:1.613319545867853e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0332990465685725e-05 - rollout_corr/rollout_is_max:3.7719850540161133 - rollout_corr/rollout_is_min:0.08076421916484833 - rollout_corr/rollout_is_std:0.030701594427227974 - rollout_corr/rollout_is_eff_sample_size:0.9990586567374643 - rollout_corr/rollout_is_seq_mean:1.000076413154602 - rollout_corr/rollout_is_seq_std:0.0018174588913097978 - rollout_corr/rollout_is_seq_max:1.0099468231201172 - rollout_corr/rollout_is_seq_min:0.9935795664787292 - rollout_corr/rollout_is_seq_max_deviation:0.009946823120117188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0983206499367952) - rollout_corr/training_log_ppl:np.float64(0.091351964041678) - rollout_corr/kl:np.float64(0.0005401147962658115) - rollout_corr/k3_kl:np.float64(0.0005354781641990769) - rollout_corr/rollout_ppl:np.float64(1.0977034606039524) - rollout_corr/rollout_log_ppl:np.float64(0.09081184863316594) - rollout_corr/log_ppl_diff:np.float64(0.0005401154085120652) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013354642323974986) - rollout_corr/log_ppl_diff_max:np.float64(0.010618716478347778) - rollout_corr/log_ppl_diff_min:np.float64(-0.006194006651639938) - rollout_corr/ppl_ratio:np.float64(1.000541973626241) - rollout_corr/chi2_token:np.float64(0.0011658316943794489) - rollout_corr/chi2_seq:np.float64(0.8551819822750986) - actor/pg_loss:np.float64(0.00199379853438586) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002786078871395148) - actor/ppo_kl:np.float64(0.00011415650995871829) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(42518.53975814115) - self_distillation/token_reweight_w_std:np.float64(780929.215723624) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994724853895605) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03163288858195301) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.21826045215129852) - perf/mfu/actor:np.float64(0.04429235206402637) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.40694427490234) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01199503056704998 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01199503056704998 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:484.25 - response_length/max:1568.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.25 - response_length_non_aborted/max:1568.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6304274797439575e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7051511611789465) - timing_s/agent_loop/generate_sequences/max:np.float64(11.325649071484804) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.876759573868185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.325649071484804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1568 - timing_s/gen:17.214315151795745 - timing_s/reward:0.06438344530761242 - timing_s/old_log_prob:2.4776456244289875 - timing_s/adv:0.5917312819510698 - timing_s/update_actor:15.933018373325467 - timing_s/step:36.368578501045704 - timing_s/stop_profile:4.634261131286621e-05 - timing_per_token_ms/update_actor:0.08174132143097408 - timing_per_token_ms/adv:0.003035764836605119 - timing_per_token_ms/gen:0.13886095727765024 - perf/total_num_tokens:194920 - perf/time_per_step:36.368578501045704 - perf/throughput:669.946448396366 (TaskRunner pid=2074004) Training Progress: 44%|████▍ | 44/100 [38:17<41:47, 44.78s/it] (TaskRunner pid=2074004) step:45 - global_seqlen/min:18847 - global_seqlen/max:30065 - global_seqlen/minmax_diff:11218 - global_seqlen/balanced_min:23771 - global_seqlen/balanced_max:23792 - global_seqlen/mean:23779.375 - actor/entropy:0.11290297657251358 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7185672521591187 - training/rollout_probs_diff_mean:0.0027959917206317186 - training/rollout_probs_diff_std:0.011578291654586792 - training/rollout_actor_probs_pearson_corr:0.9977681040763855 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000535249710083 - rollout_corr/rollout_is_ratio_fraction_high:8.408802386838943e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0001177232334157452 - rollout_corr/rollout_is_max:2.5759212970733643 - rollout_corr/rollout_is_min:0.01617681048810482 - rollout_corr/rollout_is_std:0.031018352136015892 - rollout_corr/rollout_is_eff_sample_size:0.9990390246046145 - rollout_corr/rollout_is_seq_mean:1.0000488758087158 - rollout_corr/rollout_is_seq_std:0.001919785514473915 - rollout_corr/rollout_is_seq_max:1.009157657623291 - rollout_corr/rollout_is_seq_min:0.9913671612739563 - rollout_corr/rollout_is_seq_max_deviation:0.009157657623291016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1144616291858256) - rollout_corr/training_log_ppl:np.float64(0.10538233329680224) - rollout_corr/kl:np.float64(0.0005276777127427579) - rollout_corr/k3_kl:np.float64(0.000598768252578985) - rollout_corr/rollout_ppl:np.float64(1.1138749476522207) - rollout_corr/rollout_log_ppl:np.float64(0.1048546542024269) - rollout_corr/log_ppl_diff:np.float64(0.000527679094375344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015542648143309634) - rollout_corr/log_ppl_diff_max:np.float64(0.010761316865682602) - rollout_corr/log_ppl_diff_min:np.float64(-0.007235199213027954) - rollout_corr/ppl_ratio:np.float64(1.000530268996954) - rollout_corr/chi2_token:np.float64(0.001289344858378172) - rollout_corr/chi2_seq:np.float64(0.600404967321083) - actor/pg_loss:np.float64(0.003451700205914676) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003801528589519876) - actor/ppo_kl:np.float64(-1.3993248870747266e-07) - actor/pg_clipfrac_lower:np.float64(1.1601573987718439e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(70150.05770411994) - self_distillation/token_reweight_w_std:np.float64(1347604.018264451) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003975154832006) - self_distillation/token_reweight_w_clip_frac:np.float64(0.047006876673549414) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.3081394359469414) - perf/mfu/actor:np.float64(0.04282063737580456) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.50539779663086) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011149020865559578 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011149020865559578 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.54296875 - response_length/max:1638.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:464.54296875 - response_length_non_aborted/max:1638.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003325045108795166 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6967908591032028) - timing_s/agent_loop/generate_sequences/max:np.float64(11.396734926849604) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.698892642998544) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.396734926849604) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:1638 - timing_s/gen:17.557129818946123 - timing_s/reward:0.05820976942777634 - timing_s/old_log_prob:2.601362394168973 - timing_s/adv:0.6350415386259556 - timing_s/update_actor:16.227487675845623 - timing_s/step:37.15261420793831 - timing_s/stop_profile:0.00015691295266151428 - timing_per_token_ms/update_actor:0.08530232436641849 - timing_per_token_ms/adv:0.003338195067290223 - timing_per_token_ms/gen:0.1476344342048731 - perf/total_num_tokens:190235 - perf/time_per_step:37.15261420793831 - perf/throughput:640.0458085374546 (TaskRunner pid=2074004) Training Progress: 45%|████▌ | 45/100 [38:55<39:05, 42.64s/it] (TaskRunner pid=2074004) step:46 - global_seqlen/min:14903 - global_seqlen/max:28076 - global_seqlen/minmax_diff:13173 - global_seqlen/balanced_min:20663 - global_seqlen/balanced_max:20685 - global_seqlen/mean:20676.625 - actor/entropy:0.09572755545377731 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351487040519714 - training/rollout_probs_diff_mean:0.0026079132221639156 - training/rollout_probs_diff_std:0.01122567243874073 - training/rollout_actor_probs_pearson_corr:0.9975275993347168 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0001404285430908 - rollout_corr/rollout_is_ratio_fraction_high:3.958632441936061e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.9379482991062105e-05 - rollout_corr/rollout_is_max:5.243876934051514 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.030579039826989174 - rollout_corr/rollout_is_eff_sample_size:0.9990660338595477 - rollout_corr/rollout_is_seq_mean:1.0001130104064941 - rollout_corr/rollout_is_seq_std:0.002018552739173174 - rollout_corr/rollout_is_seq_max:1.0071282386779785 - rollout_corr/rollout_is_seq_min:0.9935094714164734 - rollout_corr/rollout_is_seq_max_deviation:0.007128238677978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0899349767714739) - rollout_corr/training_log_ppl:np.float64(0.08403534088211018) - rollout_corr/kl:np.float64(0.0005288002676930859) - rollout_corr/k3_kl:np.float64(0.0005856861293978) - rollout_corr/rollout_ppl:np.float64(1.089340966194868) - rollout_corr/rollout_log_ppl:np.float64(0.08350654012610903) - rollout_corr/log_ppl_diff:np.float64(0.0005288007560011465) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017128633153333794) - rollout_corr/log_ppl_diff_max:np.float64(0.009660359472036362) - rollout_corr/log_ppl_diff_min:np.float64(-0.006222076714038849) - rollout_corr/ppl_ratio:np.float64(1.000531758647412) - rollout_corr/chi2_token:np.float64(0.0013514722231775522) - rollout_corr/chi2_seq:np.float64(0.6378451467026025) - actor/pg_loss:np.float64(0.001488825073465705) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003111429728051007) - actor/ppo_kl:np.float64(0.00012299883569388825) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(29391.911929843947) - self_distillation/token_reweight_w_std:np.float64(617711.6287392898) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997125186491758) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02729684312362224) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.15495813265442848) - perf/mfu/actor:np.float64(0.03730698959695489) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.52793884277344) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007471918594092131 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007471918594092131 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:394.70703125 - response_length/max:1505.0 - response_length/min:67.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.70703125 - response_length_non_aborted/max:1505.0 - response_length_non_aborted/min:67.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011683255434036255 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251722864806652) - timing_s/agent_loop/generate_sequences/max:np.float64(10.226146340370178) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7836934844599455) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.226146340370178) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1505 - timing_s/gen:16.00960760936141 - timing_s/reward:0.0563168041408062 - timing_s/old_log_prob:2.468111554160714 - timing_s/adv:0.5863060262054205 - timing_s/update_actor:15.916377088055015 - timing_s/step:35.136502992361784 - timing_s/stop_profile:0.00013731792569160461 - timing_per_token_ms/update_actor:0.09622204474893155 - timing_per_token_ms/adv:0.0035444978702122596 - timing_per_token_ms/gen:0.1584403741833976 - perf/total_num_tokens:165413 - perf/time_per_step:35.136502992361784 - perf/throughput:588.4656479472311 (TaskRunner pid=2074004) Training Progress: 46%|████▌ | 46/100 [39:30<36:21, 40.40s/it] (TaskRunner pid=2074004) step:47 - global_seqlen/min:18667 - global_seqlen/max:30924 - global_seqlen/minmax_diff:12257 - global_seqlen/balanced_min:24432 - global_seqlen/balanced_max:24607 - global_seqlen/mean:24464.125 - actor/entropy:0.093343086540699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28589993715286255 - training/rollout_probs_diff_mean:0.0024511476512998343 - training/rollout_probs_diff_std:0.010416332632303238 - training/rollout_actor_probs_pearson_corr:0.997839629650116 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000395774841309 - rollout_corr/rollout_is_ratio_fraction_high:1.5632695067324676e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.471443000715226e-05 - rollout_corr/rollout_is_max:3.1934804916381836 - rollout_corr/rollout_is_min:0.2297939509153366 - rollout_corr/rollout_is_std:0.028684185817837715 - rollout_corr/rollout_is_eff_sample_size:0.9991780129114268 - rollout_corr/rollout_is_seq_mean:0.9999590516090393 - rollout_corr/rollout_is_seq_std:0.0018708055140450597 - rollout_corr/rollout_is_seq_max:1.0055705308914185 - rollout_corr/rollout_is_seq_min:0.9900708198547363 - rollout_corr/rollout_is_seq_max_deviation:0.009929180145263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.095050911884755) - rollout_corr/training_log_ppl:np.float64(0.08821879087918205) - rollout_corr/kl:np.float64(0.00046318759317642844) - rollout_corr/k3_kl:np.float64(0.0005760851533125333) - rollout_corr/rollout_ppl:np.float64(1.0945486696437001) - rollout_corr/rollout_log_ppl:np.float64(0.0877556023770012) - rollout_corr/log_ppl_diff:np.float64(0.0004631885021808557) - rollout_corr/log_ppl_abs_diff:np.float64(0.001563526107929647) - rollout_corr/log_ppl_diff_max:np.float64(0.01143985241651535) - rollout_corr/log_ppl_diff_min:np.float64(-0.00824648141860962) - rollout_corr/ppl_ratio:np.float64(1.0004658417310566) - rollout_corr/chi2_token:np.float64(0.001373328734189272) - rollout_corr/chi2_seq:np.float64(0.9047737107612193) - actor/pg_loss:np.float64(0.0018651984864845872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004548156155124161) - actor/ppo_kl:np.float64(-4.627550882929654e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(51240.47999361623) - self_distillation/token_reweight_w_std:np.float64(1059937.9959096338) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991865828633308) - self_distillation/token_reweight_w_clip_frac:np.float64(0.035300534596899524) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.20760727860033512) - perf/mfu/actor:np.float64(0.04301014037057002) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.6991195678711) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010366430506110191 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010366430506110191 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:499.75390625 - response_length/max:2319.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.75390625 - response_length_non_aborted/max:2319.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.209476411342621e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8284339234232903) - timing_s/agent_loop/generate_sequences/max:np.float64(14.41846733354032) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.81055764901248) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.41846733354032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:2319 - timing_s/gen:20.333935126662254 - timing_s/reward:0.06512773968279362 - timing_s/old_log_prob:2.491063568741083 - timing_s/adv:0.754039952531457 - timing_s/update_actor:16.0744774825871 - timing_s/step:39.806080570444465 - timing_s/stop_profile:0.00010729581117630005 - timing_per_token_ms/update_actor:0.08213290625858834 - timing_per_token_ms/adv:0.0038527841918087045 - timing_per_token_ms/gen:0.1589370950285082 - perf/total_num_tokens:195713 - perf/time_per_step:39.806080570444465 - perf/throughput:614.5826127419417 (TaskRunner pid=2074004) Training Progress: 47%|████▋ | 47/100 [40:10<35:32, 40.24s/it] (TaskRunner pid=2074004) step:48 - global_seqlen/min:14563 - global_seqlen/max:26256 - global_seqlen/minmax_diff:11693 - global_seqlen/balanced_min:21802 - global_seqlen/balanced_max:21832 - global_seqlen/mean:21823.375 - actor/entropy:0.08697269856929779 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8820101022720337 - training/rollout_probs_diff_mean:0.0027279374189674854 - training/rollout_probs_diff_std:0.01222990918904543 - training/rollout_actor_probs_pearson_corr:0.9968819618225098 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0002572536468506 - rollout_corr/rollout_is_ratio_fraction_high:4.7129349695751444e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013196218060329556 - rollout_corr/rollout_is_max:2.5243570804595947 - rollout_corr/rollout_is_min:0.09723177552223206 - rollout_corr/rollout_is_std:0.03248079866170883 - rollout_corr/rollout_is_eff_sample_size:0.9989465854769791 - rollout_corr/rollout_is_seq_mean:1.0003472566604614 - rollout_corr/rollout_is_seq_std:0.002232569269835949 - rollout_corr/rollout_is_seq_max:1.0138044357299805 - rollout_corr/rollout_is_seq_min:0.9910656213760376 - rollout_corr/rollout_is_seq_max_deviation:0.013804435729980469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0930587616749108) - rollout_corr/training_log_ppl:np.float64(0.08755744152949774) - rollout_corr/kl:np.float64(0.0005523046467796888) - rollout_corr/k3_kl:np.float64(0.00072532679760684) - rollout_corr/rollout_ppl:np.float64(1.0924336458556354) - rollout_corr/rollout_log_ppl:np.float64(0.0870051367310225) - rollout_corr/log_ppl_diff:np.float64(0.0005523047984752338) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017461000934417825) - rollout_corr/log_ppl_diff_max:np.float64(0.012880250811576843) - rollout_corr/log_ppl_diff_min:np.float64(-0.008728347718715668) - rollout_corr/ppl_ratio:np.float64(1.0005556710530072) - rollout_corr/chi2_token:np.float64(0.0017671133391559124) - rollout_corr/chi2_seq:np.float64(0.6951510151848197) - actor/pg_loss:np.float64(0.002507986268028617) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006494858009773452) - actor/ppo_kl:np.float64(0.0003249875776454303) - actor/pg_clipfrac_lower:np.float64(4.164445726928534e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(161986.24940627953) - self_distillation/token_reweight_w_std:np.float64(2150402.851084696) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997133247088641) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04499576549278572) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.378052793443203) - perf/mfu/actor:np.float64(0.04008844180647516) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.73404788970947) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010018521919846535 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010018521919846535 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.41796875 - response_length/max:1405.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.41796875 - response_length_non_aborted/max:1405.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017635710537433624 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7887936737388372) - timing_s/agent_loop/generate_sequences/max:np.float64(9.995722759515047) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.310525228465849) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.995722759515047) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1405 - timing_s/gen:16.018609369173646 - timing_s/reward:0.06094110943377018 - timing_s/old_log_prob:2.350804327055812 - timing_s/adv:0.5859208330512047 - timing_s/update_actor:15.523810938000679 - timing_s/step:34.628727208822966 - timing_s/stop_profile:0.00015043653547763824 - timing_per_token_ms/update_actor:0.08891733598721943 - timing_per_token_ms/adv:0.0033560392987519384 - timing_per_token_ms/gen:0.15098933339466727 - perf/total_num_tokens:174587 - perf/time_per_step:34.628727208822966 - perf/throughput:630.2101393561956 (TaskRunner pid=2074004) Training Progress: 48%|████▊ | 48/100 [40:45<33:25, 38.57s/it] (TaskRunner pid=2074004) step:49 - global_seqlen/min:15971 - global_seqlen/max:23148 - global_seqlen/minmax_diff:7177 - global_seqlen/balanced_min:19974 - global_seqlen/balanced_max:20001 - global_seqlen/mean:19989.25 - actor/entropy:0.08722288906574249 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4621162414550781 - training/rollout_probs_diff_mean:0.0027316466439515352 - training/rollout_probs_diff_std:0.012118976563215256 - training/rollout_actor_probs_pearson_corr:0.9968992471694946 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:1.0001474618911743 - rollout_corr/rollout_is_ratio_fraction_high:4.222527059027925e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.500686428509653e-05 - rollout_corr/rollout_is_max:2.718278646469116 - rollout_corr/rollout_is_min:0.33812370896339417 - rollout_corr/rollout_is_std:0.03343567997217178 - rollout_corr/rollout_is_eff_sample_size:0.9988835415737872 - rollout_corr/rollout_is_seq_mean:1.0000269412994385 - rollout_corr/rollout_is_seq_std:0.0024658842012286186 - rollout_corr/rollout_is_seq_max:1.0119198560714722 - rollout_corr/rollout_is_seq_min:0.9833274483680725 - rollout_corr/rollout_is_seq_max_deviation:0.01667255163192749 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0898574301972985) - rollout_corr/training_log_ppl:np.float64(0.084673610603204) - rollout_corr/kl:np.float64(0.0007291726103133556) - rollout_corr/k3_kl:np.float64(0.0008918259389361083) - rollout_corr/rollout_ppl:np.float64(1.0890288641676307) - rollout_corr/rollout_log_ppl:np.float64(0.08394443713768851) - rollout_corr/log_ppl_diff:np.float64(0.0007291734655154869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020636038680095226) - rollout_corr/log_ppl_diff_max:np.float64(0.021112143993377686) - rollout_corr/log_ppl_diff_min:np.float64(-0.007896922528743744) - rollout_corr/ppl_ratio:np.float64(1.0007340346928686) - rollout_corr/chi2_token:np.float64(0.002316090976819396) - rollout_corr/chi2_seq:np.float64(1.0663579376414418) - actor/pg_loss:np.float64(0.0028226502472534776) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006949842643280135) - actor/ppo_kl:np.float64(0.00011967911359889172) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(197467.6685416242) - self_distillation/token_reweight_w_std:np.float64(2992394.127208988) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000012104865164) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04289876340772025) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.39689869433641434) - perf/mfu/actor:np.float64(0.03667143020489837) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.6230697631836) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0072983745485544205 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0072983745485544205 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.0390625 - response_length/max:1377.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.0390625 - response_length_non_aborted/max:1377.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011089816689491272 - timing_s/agent_loop/generate_sequences/min:np.float64(0.894102580845356) - timing_s/agent_loop/generate_sequences/max:np.float64(9.03606103733182) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6704066668535233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.03606103733182) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1377 - timing_s/gen:15.202601235359907 - timing_s/reward:0.05952438898384571 - timing_s/old_log_prob:2.3819591104984283 - timing_s/adv:0.6096546649932861 - timing_s/update_actor:15.689758809283376 - timing_s/step:34.03324189968407 - timing_s/stop_profile:0.0001423526555299759 - timing_per_token_ms/update_actor:0.09811372868719047 - timing_per_token_ms/adv:0.0038123908162717842 - timing_per_token_ms/gen:0.16048349240325036 - perf/total_num_tokens:159914 - perf/time_per_step:34.03324189968407 - perf/throughput:587.34486884676 (TaskRunner pid=2074004) Training Progress: 49%|████▉ | 49/100 [41:19<31:38, 37.22s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2074004) step:50 - global_seqlen/min:18143 - global_seqlen/max:30858 - global_seqlen/minmax_diff:12715 - global_seqlen/balanced_min:24044 - global_seqlen/balanced_max:26361 - global_seqlen/mean:24342.0 - actor/entropy:0.10675854980945587 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083586573600769 - training/rollout_probs_diff_mean:0.0026588926557451487 - training/rollout_probs_diff_std:0.011075353249907494 - training/rollout_actor_probs_pearson_corr:0.9978457689285278 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9999751448631287 - rollout_corr/rollout_is_ratio_fraction_high:2.371916525589768e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.115749758668244e-05 - rollout_corr/rollout_is_max:3.3773059844970703 - rollout_corr/rollout_is_min:0.2692754864692688 - rollout_corr/rollout_is_std:0.030405066907405853 - rollout_corr/rollout_is_eff_sample_size:0.9990761478330206 - rollout_corr/rollout_is_seq_mean:0.9999284744262695 - rollout_corr/rollout_is_seq_std:0.002185159595683217 - rollout_corr/rollout_is_seq_max:1.0124692916870117 - rollout_corr/rollout_is_seq_min:0.9912188649177551 - rollout_corr/rollout_is_seq_max_deviation:0.012469291687011719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1083718882873654) - rollout_corr/training_log_ppl:np.float64(0.10039229939866345) - rollout_corr/kl:np.float64(0.0004886855121668532) - rollout_corr/k3_kl:np.float64(0.0005853448297905572) - rollout_corr/rollout_ppl:np.float64(1.1078288545832038) - rollout_corr/rollout_log_ppl:np.float64(0.09990361310701701) - rollout_corr/log_ppl_diff:np.float64(0.0004886862916464452) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016839853378769476) - rollout_corr/log_ppl_diff_max:np.float64(0.010547831654548645) - rollout_corr/log_ppl_diff_min:np.float64(-0.011270195245742798) - rollout_corr/ppl_ratio:np.float64(1.000491572311148) - rollout_corr/chi2_token:np.float64(0.0013913614675402641) - rollout_corr/chi2_seq:np.float64(1.0534377871081233) - actor/pg_loss:np.float64(0.0014425351982936263) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040900465501181316) - actor/ppo_kl:np.float64(-0.00011984414344468064) - actor/pg_clipfrac_lower:np.float64(3.814697265625e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(121532.27847940498) - self_distillation/token_reweight_w_std:np.float64(2120513.0500425613) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004689684137702) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03253905474412022) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.20551946759223938) - perf/mfu/actor:np.float64(0.04243176899328058) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7705078125) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/reward/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07946446968361667) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8169375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.047402393710538074) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6619124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05008887435519682) - val-aux/sciknoweval/score/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/score/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07946446968361667) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8169375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.047402393710538074) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6619124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05008887435519682) - val-core/sciknoweval/acc/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/acc/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07946446968361667) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8169375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.047402393710538074) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6619124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05008887435519682) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.026059282360820557) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9991375000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.008110749235580297) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.985425) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03423627902458588) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.992475) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.025689990721416346) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.000683625445693766) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9733875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04275106884274511) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9975375) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.014926669851977312) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.954525) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.047782282724323125) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996125000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005601383627055409) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9313375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04368822856243866) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00039508701573197775) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0210300050675869 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0210300050675869 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:494.0625 - response_length/max:4102.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:494.0625 - response_length_non_aborted/max:4102.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.833462536334991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6969410367310047) - timing_s/agent_loop/generate_sequences/max:np.float64(22.719568610191345) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.79131419971236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.719568610191345) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:4102 - timing_s/gen:28.734515454620123 - timing_s/reward:0.06417730078101158 - timing_s/old_log_prob:2.4680988006293774 - timing_s/adv:0.8539764694869518 - timing_s/update_actor:16.26316487416625 - timing_s/step:48.47273708879948 - timing_s/testing:90.87557928822935 - timing_s/save_checkpoint:6.687707267701626 - timing_s/stop_profile:0.0001324862241744995 - timing_per_token_ms/update_actor:0.08351391049506127 - timing_per_token_ms/adv:0.004385303536515856 - timing_per_token_ms/gen:0.22718623857226536 - perf/total_num_tokens:194736 - perf/time_per_step:48.47273708879948 - perf/throughput:502.17919312884584 (TaskRunner pid=2074004) Training Progress: 50%|█████ | 50/100 [43:45<58:14, 69.89s/it] (TaskRunner pid=2074004) step:51 - global_seqlen/min:16973 - global_seqlen/max:21967 - global_seqlen/minmax_diff:4994 - global_seqlen/balanced_min:19550 - global_seqlen/balanced_max:19572 - global_seqlen/mean:19567.375 - actor/entropy:0.11901509016752243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083246231079102 - training/rollout_probs_diff_mean:0.003160488326102495 - training/rollout_probs_diff_std:0.012160852551460266 - training/rollout_actor_probs_pearson_corr:0.9976589679718018 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001776218414307 - rollout_corr/rollout_is_ratio_fraction_high:5.6728575145825744e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.807428871979937e-05 - rollout_corr/rollout_is_max:2.820579767227173 - rollout_corr/rollout_is_min:0.30462947487831116 - rollout_corr/rollout_is_std:0.03463872894644737 - rollout_corr/rollout_is_eff_sample_size:0.9988020720902324 - rollout_corr/rollout_is_seq_mean:1.000294804573059 - rollout_corr/rollout_is_seq_std:0.0022349499631673098 - rollout_corr/rollout_is_seq_max:1.0152208805084229 - rollout_corr/rollout_is_seq_min:0.995274543762207 - rollout_corr/rollout_is_seq_max_deviation:0.015220880508422852 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1107426239177585) - rollout_corr/training_log_ppl:np.float64(0.10169691975897877) - rollout_corr/kl:np.float64(0.0001784627847740694) - rollout_corr/k3_kl:np.float64(0.0006646001246366495) - rollout_corr/rollout_ppl:np.float64(1.1105284076184034) - rollout_corr/rollout_log_ppl:np.float64(0.10151845622021938) - rollout_corr/log_ppl_diff:np.float64(0.00017846353875938803) - rollout_corr/log_ppl_abs_diff:np.float64(0.001676651110756211) - rollout_corr/log_ppl_diff_max:np.float64(0.006288684904575348) - rollout_corr/log_ppl_diff_min:np.float64(-0.010029513388872147) - rollout_corr/ppl_ratio:np.float64(1.0001808677334338) - rollout_corr/chi2_token:np.float64(0.002416089177131653) - rollout_corr/chi2_seq:np.float64(2.1752094991970807) - actor/pg_loss:np.float64(0.0023010852746665478) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047835509440119495) - actor/ppo_kl:np.float64(-0.0001440780888302129) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(79808.69256585464) - self_distillation/token_reweight_w_std:np.float64(1330398.761181703) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998520778026432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04437096892797854) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.2404463179409504) - perf/mfu/actor:np.float64(0.03516023093043365) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7251968383789) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004383700899779797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004383700899779797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.29296875 - response_length/max:1233.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.29296875 - response_length_non_aborted/max:1233.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.046704947948456e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8948994744569063) - timing_s/agent_loop/generate_sequences/max:np.float64(8.36026605591178) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4736403662464) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.36026605591178) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1233 - timing_s/gen:14.1562640350312 - timing_s/reward:0.051813943311572075 - timing_s/old_log_prob:2.4304460305720568 - timing_s/adv:0.6007976606488228 - timing_s/update_actor:15.809248071163893 - timing_s/step:33.15378465689719 - timing_s/stop_profile:0.00011606328189373016 - timing_per_token_ms/update_actor:0.10099239212697086 - timing_per_token_ms/adv:0.0038380062517891565 - timing_per_token_ms/gen:0.1606129413203145 - perf/total_num_tokens:156539 - perf/time_per_step:33.15378465689719 - perf/throughput:590.2003407001462 (TaskRunner pid=2074004) Training Progress: 51%|█████ | 51/100 [44:18<48:05, 58.88s/it] (TaskRunner pid=2074004) step:52 - global_seqlen/min:16957 - global_seqlen/max:26308 - global_seqlen/minmax_diff:9351 - global_seqlen/balanced_min:22416 - global_seqlen/balanced_max:22423 - global_seqlen/mean:22420.25 - actor/entropy:0.11742626130580902 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6386399269104004 - training/rollout_probs_diff_mean:0.0029321101028472185 - training/rollout_probs_diff_std:0.011663330718874931 - training/rollout_actor_probs_pearson_corr:0.9977625608444214 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:0.9999903440475464 - rollout_corr/rollout_is_ratio_fraction_high:3.6525008908938617e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.305001781787723e-05 - rollout_corr/rollout_is_max:3.6345746517181396 - rollout_corr/rollout_is_min:0.252546101808548 - rollout_corr/rollout_is_std:0.03316900134086609 - rollout_corr/rollout_is_eff_sample_size:0.9989007886433641 - rollout_corr/rollout_is_seq_mean:1.0000351667404175 - rollout_corr/rollout_is_seq_std:0.002520139329135418 - rollout_corr/rollout_is_seq_max:1.0145207643508911 - rollout_corr/rollout_is_seq_min:0.9895867109298706 - rollout_corr/rollout_is_seq_max_deviation:0.014520764350891113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1248778235167265) - rollout_corr/training_log_ppl:np.float64(0.11452079268201487) - rollout_corr/kl:np.float64(0.0006545804253192955) - rollout_corr/k3_kl:np.float64(0.0007203538166162815) - rollout_corr/rollout_ppl:np.float64(1.1241182647645473) - rollout_corr/rollout_log_ppl:np.float64(0.11386621197380009) - rollout_corr/log_ppl_diff:np.float64(0.0006545807082147803) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016909198930079583) - rollout_corr/log_ppl_diff_max:np.float64(0.011592693626880646) - rollout_corr/log_ppl_diff_min:np.float64(-0.010950211435556412) - rollout_corr/ppl_ratio:np.float64(1.0006579770706594) - rollout_corr/chi2_token:np.float64(0.0015927678905427456) - rollout_corr/chi2_seq:np.float64(0.5052446881309152) - actor/pg_loss:np.float64(0.0011261987965554) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031590935577696655) - actor/ppo_kl:np.float64(4.731501897392576e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(120173.51662478969) - self_distillation/token_reweight_w_std:np.float64(1645237.523866353) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993055597878993) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026213745499262586) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.3443529047071934) - perf/mfu/actor:np.float64(0.04109087151546889) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.80995559692383) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003660490969195962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003660490969195962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:427.7890625 - response_length/max:1240.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.7890625 - response_length_non_aborted/max:1240.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.612925350666046e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6170574445277452) - timing_s/agent_loop/generate_sequences/max:np.float64(9.085840474814177) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.157990079183946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.085840474814177) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1240 - timing_s/gen:14.695505177602172 - timing_s/reward:0.06002468429505825 - timing_s/old_log_prob:2.512907398864627 - timing_s/adv:0.5510312523692846 - timing_s/update_actor:15.702309988439083 - timing_s/step:33.61664153076708 - timing_s/stop_profile:0.00011726468801498413 - timing_per_token_ms/update_actor:0.08754535513898754 - timing_per_token_ms/adv:0.0030721738850441266 - timing_per_token_ms/gen:0.13418837023213628 - perf/total_num_tokens:179362 - perf/time_per_step:33.61664153076708 - perf/throughput:666.9390212428042 (TaskRunner pid=2074004) Training Progress: 52%|█████▏ | 52/100 [44:52<41:03, 51.32s/it] (TaskRunner pid=2074004) step:53 - global_seqlen/min:15926 - global_seqlen/max:32544 - global_seqlen/minmax_diff:16618 - global_seqlen/balanced_min:21720 - global_seqlen/balanced_max:28240 - global_seqlen/mean:22614.625 - actor/entropy:0.09381777793169022 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6157671809196472 - training/rollout_probs_diff_mean:0.0025205647107213736 - training/rollout_probs_diff_std:0.011054250411689281 - training/rollout_actor_probs_pearson_corr:0.9975994229316711 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000879764556885 - rollout_corr/rollout_is_ratio_fraction_high:4.539635483524762e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.26341677363962e-05 - rollout_corr/rollout_is_max:2.6310083866119385 - rollout_corr/rollout_is_min:0.3212862014770508 - rollout_corr/rollout_is_std:0.029877066612243652 - rollout_corr/rollout_is_eff_sample_size:0.999108513924274 - rollout_corr/rollout_is_seq_mean:1.0001354217529297 - rollout_corr/rollout_is_seq_std:0.0017414779867976904 - rollout_corr/rollout_is_seq_max:1.0066789388656616 - rollout_corr/rollout_is_seq_min:0.9954082369804382 - rollout_corr/rollout_is_seq_max_deviation:0.006678938865661621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.098151988349855) - rollout_corr/training_log_ppl:np.float64(0.09104150990060589) - rollout_corr/kl:np.float64(0.00047328945871338846) - rollout_corr/k3_kl:np.float64(0.0005634092783721734) - rollout_corr/rollout_ppl:np.float64(1.0976255191490054) - rollout_corr/rollout_log_ppl:np.float64(0.09056821986359864) - rollout_corr/log_ppl_diff:np.float64(0.0004732900370072457) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015623152985426714) - rollout_corr/log_ppl_diff_max:np.float64(0.005912806838750839) - rollout_corr/log_ppl_diff_min:np.float64(-0.005394577980041504) - rollout_corr/ppl_ratio:np.float64(1.0004753414541483) - rollout_corr/chi2_token:np.float64(0.0013314660172909498) - rollout_corr/chi2_seq:np.float64(0.4222771213389933) - actor/pg_loss:np.float64(0.002000119653530419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005868266684956325) - actor/ppo_kl:np.float64(0.00012776876268461024) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(53422.7099313552) - self_distillation/token_reweight_w_std:np.float64(849382.4181836921) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993473722133785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037509667774429545) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.31460506841540337) - perf/mfu/actor:np.float64(0.03860267849970589) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.8161849975586) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.057434335350990295 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.057434335350990295 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:430.23828125 - response_length/max:8192.0 - response_length/min:95.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:430.23828125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011334381997585297 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0902089104056358) - timing_s/agent_loop/generate_sequences/max:np.float64(50.08467713370919) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.101931656085071) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.08467713370919) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.888266975060105 - timing_s/reward:0.061832696199417114 - timing_s/old_log_prob:2.569713741540909 - timing_s/adv:0.5755481589585543 - timing_s/update_actor:16.975983068346977 - timing_s/step:76.15641017071903 - timing_s/stop_profile:2.802535891532898e-05 - timing_per_token_ms/update_actor:0.09383299009129588 - timing_per_token_ms/adv:0.003181282902980672 - timing_per_token_ms/gen:0.5074247280763757 - perf/total_num_tokens:180917 - perf/time_per_step:76.15641017071903 - perf/throughput:296.94972424914766 (TaskRunner pid=2074004) Training Progress: 53%|█████▎ | 53/100 [46:08<46:02, 58.79s/it] (TaskRunner pid=2074004) step:54 - global_seqlen/min:16107 - global_seqlen/max:24016 - global_seqlen/minmax_diff:7909 - global_seqlen/balanced_min:19289 - global_seqlen/balanced_max:19501 - global_seqlen/mean:19320.75 - actor/entropy:0.09417933225631714 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39961040019989014 - training/rollout_probs_diff_mean:0.0026771980337798595 - training/rollout_probs_diff_std:0.011605466715991497 - training/rollout_actor_probs_pearson_corr:0.9973964095115662 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0001647472381592 - rollout_corr/rollout_is_ratio_fraction_high:1.1300201549602207e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.040161239681765e-05 - rollout_corr/rollout_is_max:2.7947683334350586 - rollout_corr/rollout_is_min:0.2384055107831955 - rollout_corr/rollout_is_std:0.03145531937479973 - rollout_corr/rollout_is_eff_sample_size:0.9990118978361671 - rollout_corr/rollout_is_seq_mean:1.0001072883605957 - rollout_corr/rollout_is_seq_std:0.0022169763687998056 - rollout_corr/rollout_is_seq_max:1.009650707244873 - rollout_corr/rollout_is_seq_min:0.9886189699172974 - rollout_corr/rollout_is_seq_max_deviation:0.011381030082702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0925864586606622) - rollout_corr/training_log_ppl:np.float64(0.08678890572628006) - rollout_corr/kl:np.float64(0.0009032392683874235) - rollout_corr/k3_kl:np.float64(0.0007826630679801383) - rollout_corr/rollout_ppl:np.float64(1.0915671489201486) - rollout_corr/rollout_log_ppl:np.float64(0.08588566524122143) - rollout_corr/log_ppl_diff:np.float64(0.0009032404850586317) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018698280400712974) - rollout_corr/log_ppl_diff_max:np.float64(0.03783611208200455) - rollout_corr/log_ppl_diff_min:np.float64(-0.005891826003789902) - rollout_corr/ppl_ratio:np.float64(1.0009094115812331) - rollout_corr/chi2_token:np.float64(0.0010525539983063936) - rollout_corr/chi2_seq:np.float64(0.43362018815241754) - actor/pg_loss:np.float64(0.002331349765881896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007941815301819588) - actor/ppo_kl:np.float64(0.0004760969166213158) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(237838.98983726162) - self_distillation/token_reweight_w_std:np.float64(3087923.700471521) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999815991614014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04877196221787017) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.3219018206000328) - perf/mfu/actor:np.float64(0.03530995118577071) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7523651123047) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000819264561869204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000819264561869204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.6796875 - response_length/max:1721.0 - response_length/min:64.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.6796875 - response_length_non_aborted/max:1721.0 - response_length_non_aborted/min:64.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.285946488380432e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.751097746193409) - timing_s/agent_loop/generate_sequences/max:np.float64(10.377681985497475) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.351018370616657) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.377681985497475) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1721 - timing_s/gen:16.176042480394244 - timing_s/reward:0.05808785557746887 - timing_s/old_log_prob:2.318193769082427 - timing_s/adv:0.6144152693450451 - timing_s/update_actor:15.822785630822182 - timing_s/step:35.07668521627784 - timing_s/stop_profile:0.00010946020483970642 - timing_per_token_ms/update_actor:0.10236912148093488 - timing_per_token_ms/adv:0.00397509975897057 - timing_per_token_ms/gen:0.18279253373555546 - perf/total_num_tokens:154566 - perf/time_per_step:35.07668521627784 - perf/throughput:550.8145904001764 (TaskRunner pid=2074004) Training Progress: 54%|█████▍ | 54/100 [46:43<39:37, 51.69s/it] (TaskRunner pid=2074004) step:55 - global_seqlen/min:14652 - global_seqlen/max:26201 - global_seqlen/minmax_diff:11549 - global_seqlen/balanced_min:20421 - global_seqlen/balanced_max:21183 - global_seqlen/mean:20601.125 - actor/entropy:0.085003562271595 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7348287105560303 - training/rollout_probs_diff_mean:0.0025023941416293383 - training/rollout_probs_diff_std:0.01154287438839674 - training/rollout_actor_probs_pearson_corr:0.9971373081207275 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.0001779794692993 - rollout_corr/rollout_is_ratio_fraction_high:1.0315978215658106e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013410771498456597 - rollout_corr/rollout_is_max:2.935729742050171 - rollout_corr/rollout_is_min:0.038828011602163315 - rollout_corr/rollout_is_std:0.030352089554071426 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0003106594085693 - rollout_corr/rollout_is_seq_std:0.002173425629734993 - rollout_corr/rollout_is_seq_max:1.0108247995376587 - rollout_corr/rollout_is_seq_min:0.9864917993545532 - rollout_corr/rollout_is_seq_max_deviation:0.013508200645446777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.085447907447815) - rollout_corr/training_log_ppl:np.float64(0.0803521656380326) - rollout_corr/kl:np.float64(0.0008227331833889906) - rollout_corr/k3_kl:np.float64(0.0011211537594633114) - rollout_corr/rollout_ppl:np.float64(1.0845554950647056) - rollout_corr/rollout_log_ppl:np.float64(0.0795294317031221) - rollout_corr/log_ppl_diff:np.float64(0.0008227339349105023) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020583992882166058) - rollout_corr/log_ppl_diff_max:np.float64(0.04965774714946747) - rollout_corr/log_ppl_diff_min:np.float64(-0.009343862533569336) - rollout_corr/ppl_ratio:np.float64(1.0008315925952047) - rollout_corr/chi2_token:np.float64(0.002741375006735325) - rollout_corr/chi2_seq:np.float64(0.736898449016735) - actor/pg_loss:np.float64(0.001779418089427054) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009461770737289044) - actor/ppo_kl:np.float64(0.0004943441474765109) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(220652.25802971283) - self_distillation/token_reweight_w_std:np.float64(2705884.476536175) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000388199230656) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0328305515286047) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.42648517340421677) - perf/mfu/actor:np.float64(0.037390062668015575) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.62273788452148) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006393327843397856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006393327843397856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:378.66015625 - response_length/max:2305.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.66015625 - response_length_non_aborted/max:2305.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012177042663097382 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9361808206886053) - timing_s/agent_loop/generate_sequences/max:np.float64(13.556594530120492) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.681363452458754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.556594530120492) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:318 - timing_s/agent_loop/slowest/response_length:2305 - timing_s/gen:19.531876169145107 - timing_s/reward:0.05353007838129997 - timing_s/old_log_prob:2.345628321170807 - timing_s/adv:0.5701816361397505 - timing_s/update_actor:15.82276027649641 - timing_s/step:38.36092114262283 - timing_s/stop_profile:0.00011600367724895477 - timing_per_token_ms/update_actor:0.0960066517999406 - timing_per_token_ms/adv:0.003459651087863833 - timing_per_token_ms/gen:0.20149041304295684 - perf/total_num_tokens:164809 - perf/time_per_step:38.36092114262283 - perf/throughput:537.0341583667052 (TaskRunner pid=2074004) Training Progress: 55%|█████▌ | 55/100 [47:22<35:46, 47.70s/it] (TaskRunner pid=2074004) step:56 - global_seqlen/min:16673 - global_seqlen/max:30667 - global_seqlen/minmax_diff:13994 - global_seqlen/balanced_min:21519 - global_seqlen/balanced_max:21670 - global_seqlen/mean:21542.25 - actor/entropy:0.0999220460653305 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9910814762115479 - training/rollout_probs_diff_mean:0.0026262723840773106 - training/rollout_probs_diff_std:0.011972086504101753 - training/rollout_actor_probs_pearson_corr:0.9973817467689514 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:1.0003807544708252 - rollout_corr/rollout_is_ratio_fraction_high:4.651249400922097e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.372248703381047e-05 - rollout_corr/rollout_is_max:44.50603103637695 - rollout_corr/rollout_is_min:0.0011786288814619184 - rollout_corr/rollout_is_std:0.030854584649205208 - rollout_corr/rollout_is_eff_sample_size:0.9990490190485594 - rollout_corr/rollout_is_seq_mean:1.0000252723693848 - rollout_corr/rollout_is_seq_std:0.004327365662902594 - rollout_corr/rollout_is_seq_max:1.0554637908935547 - rollout_corr/rollout_is_seq_min:0.9875951409339905 - rollout_corr/rollout_is_seq_max_deviation:0.05546379089355469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0925669269636273) - rollout_corr/training_log_ppl:np.float64(0.08702500175058958) - rollout_corr/kl:np.float64(0.0007599289466115877) - rollout_corr/k3_kl:np.float64(0.000812841809917586) - rollout_corr/rollout_ppl:np.float64(1.0917559866793454) - rollout_corr/rollout_log_ppl:np.float64(0.08626507179360487) - rollout_corr/log_ppl_diff:np.float64(0.0007599299569847062) - rollout_corr/log_ppl_abs_diff:np.float64(0.001818598095269408) - rollout_corr/log_ppl_diff_max:np.float64(0.024008244276046753) - rollout_corr/log_ppl_diff_min:np.float64(-0.014017723500728607) - rollout_corr/ppl_ratio:np.float64(1.0007652402855456) - rollout_corr/chi2_token:np.float64(0.007633969187736511) - rollout_corr/chi2_seq:np.float64(13.315856878180057) - actor/pg_loss:np.float64(0.0017298469319939613) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006172826681449806) - actor/ppo_kl:np.float64(-5.590297072188832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(123974.65548743168) - self_distillation/token_reweight_w_std:np.float64(2170329.61304679) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005807334091514) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04290506925462978) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.27438871562480927) - perf/mfu/actor:np.float64(0.03876809494412483) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.63893127441406) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024676039814949036 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024676039814949036 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:419.9140625 - response_length/max:2024.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:419.9140625 - response_length_non_aborted/max:2024.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012747198343276978 - timing_s/agent_loop/generate_sequences/min:np.float64(0.48513596691191196) - timing_s/agent_loop/generate_sequences/max:np.float64(13.075700936838984) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0640050550719025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.075700936838984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:2024 - timing_s/gen:18.986181549727917 - timing_s/reward:0.06117998994886875 - timing_s/old_log_prob:2.4063704144209623 - timing_s/adv:0.5998788680881262 - timing_s/update_actor:15.829487767070532 - timing_s/step:37.97097833454609 - timing_s/stop_profile:0.00010865926742553711 - timing_per_token_ms/update_actor:0.09185140692749441 - timing_per_token_ms/adv:0.003480827606727049 - timing_per_token_ms/gen:0.17661892825659936 - perf/total_num_tokens:172338 - perf/time_per_step:37.97097833454609 - perf/throughput:567.3346051344905 (TaskRunner pid=2074004) Training Progress: 56%|█████▌ | 56/100 [48:00<32:51, 44.80s/it] (TaskRunner pid=2074004) step:57 - global_seqlen/min:19855 - global_seqlen/max:29475 - global_seqlen/minmax_diff:9620 - global_seqlen/balanced_min:23604 - global_seqlen/balanced_max:23634 - global_seqlen/mean:23625.125 - actor/entropy:0.09015602618455887 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456105500459671 - training/rollout_probs_diff_mean:0.0024542445316910744 - training/rollout_probs_diff_std:0.011214017868041992 - training/rollout_actor_probs_pearson_corr:0.9974842667579651 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0000454187393188 - rollout_corr/rollout_is_ratio_fraction_high:5.9702000726247206e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010234629007754847 - rollout_corr/rollout_is_max:2.9357409477233887 - rollout_corr/rollout_is_min:0.22341881692409515 - rollout_corr/rollout_is_std:0.030461862683296204 - rollout_corr/rollout_is_eff_sample_size:0.9990729351369017 - rollout_corr/rollout_is_seq_mean:1.0001904964447021 - rollout_corr/rollout_is_seq_std:0.002443463308736682 - rollout_corr/rollout_is_seq_max:1.0193251371383667 - rollout_corr/rollout_is_seq_min:0.9929532408714294 - rollout_corr/rollout_is_seq_max_deviation:0.0193251371383667 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0897927153855562) - rollout_corr/training_log_ppl:np.float64(0.0843119082164776) - rollout_corr/kl:np.float64(0.00043658774763111197) - rollout_corr/k3_kl:np.float64(0.0005583141538294534) - rollout_corr/rollout_ppl:np.float64(1.0893031116575003) - rollout_corr/rollout_log_ppl:np.float64(0.0838753203752276) - rollout_corr/log_ppl_diff:np.float64(0.0004365878412500024) - rollout_corr/log_ppl_abs_diff:np.float64(0.001471972107538022) - rollout_corr/log_ppl_diff_max:np.float64(0.00870583951473236) - rollout_corr/log_ppl_diff_min:np.float64(-0.012766733765602112) - rollout_corr/ppl_ratio:np.float64(1.000438829883933) - rollout_corr/chi2_token:np.float64(0.0013955945614725351) - rollout_corr/chi2_seq:np.float64(1.2546640543732792) - actor/pg_loss:np.float64(0.0011045325081795454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035431018341114395) - actor/ppo_kl:np.float64(0.00010037352073410943) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(72716.30810955423) - self_distillation/token_reweight_w_std:np.float64(1347980.3629608657) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001339728478342) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02943878156656865) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.24138232693076134) - perf/mfu/actor:np.float64(0.0430505553899597) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.70356369018555) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008093886077404022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008093886077404022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.00390625 - response_length/max:1631.0 - response_length/min:45.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:458.00390625 - response_length_non_aborted/max:1631.0 - response_length_non_aborted/min:45.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013142824172973633 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5333198085427284) - timing_s/agent_loop/generate_sequences/max:np.float64(11.510037498548627) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.700965259689838) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.510037498548627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1631 - timing_s/gen:17.477134620770812 - timing_s/reward:0.0652174036949873 - timing_s/old_log_prob:2.3839294649660587 - timing_s/adv:0.5826396737247705 - timing_s/update_actor:15.731351995840669 - timing_s/step:36.32835609279573 - timing_s/stop_profile:0.00012176670134067535 - timing_per_token_ms/update_actor:0.08323422625192813 - timing_per_token_ms/adv:0.003082733285669232 - timing_per_token_ms/gen:0.1490599887484824 - perf/total_num_tokens:189001 - perf/time_per_step:36.32835609279573 - perf/throughput:650.3218846361477 (TaskRunner pid=2074004) Training Progress: 57%|█████▋ | 57/100 [48:36<30:17, 42.27s/it] (TaskRunner pid=2074004) step:58 - global_seqlen/min:14549 - global_seqlen/max:27391 - global_seqlen/minmax_diff:12842 - global_seqlen/balanced_min:22974 - global_seqlen/balanced_max:22989 - global_seqlen/mean:22981.5 - actor/entropy:0.10685306042432785 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6651362180709839 - training/rollout_probs_diff_mean:0.0026612752117216587 - training/rollout_probs_diff_std:0.011326903477311134 - training/rollout_actor_probs_pearson_corr:0.9977207779884338 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9998153448104858 - rollout_corr/rollout_is_ratio_fraction_high:8.753195288591087e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00012254473404027522 - rollout_corr/rollout_is_max:2.095289468765259 - rollout_corr/rollout_is_min:0.04907945916056633 - rollout_corr/rollout_is_std:0.030399184674024582 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:0.9997419118881226 - rollout_corr/rollout_is_seq_std:0.0024453294463455677 - rollout_corr/rollout_is_seq_max:1.0176780223846436 - rollout_corr/rollout_is_seq_min:0.9827814102172852 - rollout_corr/rollout_is_seq_max_deviation:0.017678022384643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1037812139838934) - rollout_corr/training_log_ppl:np.float64(0.09657903140760027) - rollout_corr/kl:np.float64(0.0008447893188705535) - rollout_corr/k3_kl:np.float64(0.0005789172635033424) - rollout_corr/rollout_ppl:np.float64(1.1028364589437842) - rollout_corr/rollout_log_ppl:np.float64(0.09573424071641057) - rollout_corr/log_ppl_diff:np.float64(0.0008447906911897007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017328100257145707) - rollout_corr/log_ppl_diff_max:np.float64(0.02989429235458374) - rollout_corr/log_ppl_diff_min:np.float64(-0.01144128292798996) - rollout_corr/ppl_ratio:np.float64(1.0008499294053763) - rollout_corr/chi2_token:np.float64(0.0006312790792435408) - rollout_corr/chi2_seq:np.float64(0.1915365003515035) - actor/pg_loss:np.float64(0.001689612865447998) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005289059029109922) - actor/ppo_kl:np.float64(3.6509948291296723e-06) - actor/pg_clipfrac_lower:np.float64(5.7276392908534035e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(124004.99291193858) - self_distillation/token_reweight_w_std:np.float64(2293706.94823437) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002958420664072) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03820525103947148) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2612778563052416) - perf/mfu/actor:np.float64(0.04187616741345731) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.75995635986328) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020541559904813766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020541559904813766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:446.265625 - response_length/max:1848.0 - response_length/min:58.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:446.265625 - response_length_non_aborted/max:1848.0 - response_length_non_aborted/min:58.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.460374712944031e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.705725435167551) - timing_s/agent_loop/generate_sequences/max:np.float64(11.959222553297877) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.448366878685192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.959222553297877) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1848 - timing_s/gen:17.897763485088944 - timing_s/reward:0.06247272342443466 - timing_s/old_log_prob:2.423390854150057 - timing_s/adv:0.5865025147795677 - timing_s/update_actor:16.00029367208481 - timing_s/step:37.05911082029343 - timing_s/stop_profile:0.00011603906750679016 - timing_per_token_ms/update_actor:0.08702811866112312 - timing_per_token_ms/adv:0.0031900796008722653 - timing_per_token_ms/gen:0.1566626123480353 - perf/total_num_tokens:183852 - perf/time_per_step:37.05911082029343 - perf/throughput:620.130906848834 (TaskRunner pid=2074004) Training Progress: 58%|█████▊ | 58/100 [49:13<28:30, 40.72s/it] (TaskRunner pid=2074004) step:59 - global_seqlen/min:16270 - global_seqlen/max:25747 - global_seqlen/minmax_diff:9477 - global_seqlen/balanced_min:21707 - global_seqlen/balanced_max:21791 - global_seqlen/mean:21721.5 - actor/entropy:0.10231564939022064 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46255624294281006 - training/rollout_probs_diff_mean:0.0026242148596793413 - training/rollout_probs_diff_std:0.011124146170914173 - training/rollout_actor_probs_pearson_corr:0.9976997971534729 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0001174211502075 - rollout_corr/rollout_is_ratio_fraction_high:3.8038724596844986e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8038724596844986e-05 - rollout_corr/rollout_is_max:2.163364887237549 - rollout_corr/rollout_is_min:2.3478437469748314e-06 - rollout_corr/rollout_is_std:0.03120991960167885 - rollout_corr/rollout_is_eff_sample_size:0.9990270077403499 - rollout_corr/rollout_is_seq_mean:1.000049114227295 - rollout_corr/rollout_is_seq_std:0.002008486306294799 - rollout_corr/rollout_is_seq_max:1.0093592405319214 - rollout_corr/rollout_is_seq_min:0.9932237863540649 - rollout_corr/rollout_is_seq_max_deviation:0.009359240531921387 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1050791395828128) - rollout_corr/training_log_ppl:np.float64(0.09638163917770726) - rollout_corr/kl:np.float64(0.0006338543649420103) - rollout_corr/k3_kl:np.float64(0.0009249671898068357) - rollout_corr/rollout_ppl:np.float64(1.1043821601197124) - rollout_corr/rollout_log_ppl:np.float64(0.09574778548085305) - rollout_corr/log_ppl_diff:np.float64(0.000633853696854203) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018177191577706253) - rollout_corr/log_ppl_diff_max:np.float64(0.02475271373987198) - rollout_corr/log_ppl_diff_min:np.float64(-0.016850709915161133) - rollout_corr/ppl_ratio:np.float64(1.0006383063737303) - rollout_corr/chi2_token:np.float64(0.002892733784392476) - rollout_corr/chi2_seq:np.float64(1.0235684728249907) - actor/pg_loss:np.float64(0.0013639027019962668) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011857084368784854) - actor/ppo_kl:np.float64(5.230466129590994e-05) - actor/pg_clipfrac_lower:np.float64(1.7208149074576795e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(187076.23620933085) - self_distillation/token_reweight_w_std:np.float64(2489852.4811536297) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00058325775899) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037732115903054364) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.295432485640049) - perf/mfu/actor:np.float64(0.03939332486847546) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.67478942871094) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005545332562178373 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005545332562178373 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:410.765625 - response_length/max:1910.0 - response_length/min:68.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:410.765625 - response_length_non_aborted/max:1910.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.881092071533203e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5705593023449183) - timing_s/agent_loop/generate_sequences/max:np.float64(11.528048625215888) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9566591325128684) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.528048625215888) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1910 - timing_s/gen:17.374146005138755 - timing_s/reward:0.0605713315308094 - timing_s/old_log_prob:2.3631235398352146 - timing_s/adv:0.5710447505116463 - timing_s/update_actor:15.808569202199578 - timing_s/step:36.26629160158336 - timing_s/stop_profile:0.00011047534644603729 - timing_per_token_ms/update_actor:0.0909730520578665 - timing_per_token_ms/adv:0.0032861724012593872 - timing_per_token_ms/gen:0.16522258363896264 - perf/total_num_tokens:173772 - perf/time_per_step:36.26629160158336 - perf/throughput:598.9446133238407 (TaskRunner pid=2074004) Training Progress: 59%|█████▉ | 59/100 [49:49<26:55, 39.40s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2074004) step:60 - global_seqlen/min:14585 - global_seqlen/max:28896 - global_seqlen/minmax_diff:14311 - global_seqlen/balanced_min:22508 - global_seqlen/balanced_max:22534 - global_seqlen/mean:22528.125 - actor/entropy:0.10657462477684021 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5907468795776367 - training/rollout_probs_diff_mean:0.0025613645557314157 - training/rollout_probs_diff_std:0.011115039698779583 - training/rollout_actor_probs_pearson_corr:0.9978196620941162 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001739263534546 - rollout_corr/rollout_is_ratio_fraction_high:3.601494609029032e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.904109901981428e-05 - rollout_corr/rollout_is_max:3.41389799118042 - rollout_corr/rollout_is_min:0.04764045029878616 - rollout_corr/rollout_is_std:0.030336376279592514 - rollout_corr/rollout_is_eff_sample_size:0.9990810264110818 - rollout_corr/rollout_is_seq_mean:1.0001338720321655 - rollout_corr/rollout_is_seq_std:0.0019507677061483264 - rollout_corr/rollout_is_seq_max:1.0104360580444336 - rollout_corr/rollout_is_seq_min:0.9874913692474365 - rollout_corr/rollout_is_seq_max_deviation:0.012508630752563477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.101980961393565) - rollout_corr/training_log_ppl:np.float64(0.09394121470904793) - rollout_corr/kl:np.float64(0.0005248432001501158) - rollout_corr/k3_kl:np.float64(0.0005477877917310359) - rollout_corr/rollout_ppl:np.float64(1.1014048550277948) - rollout_corr/rollout_log_ppl:np.float64(0.09341637074976461) - rollout_corr/log_ppl_diff:np.float64(0.0005248439592833165) - rollout_corr/log_ppl_abs_diff:np.float64(0.001438066206901567) - rollout_corr/log_ppl_diff_max:np.float64(0.013666749000549316) - rollout_corr/log_ppl_diff_min:np.float64(-0.009086042642593384) - rollout_corr/ppl_ratio:np.float64(1.0005271381232888) - rollout_corr/chi2_token:np.float64(0.0011212392710149288) - rollout_corr/chi2_seq:np.float64(0.3878027028404176) - actor/pg_loss:np.float64(0.0020158932311460376) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007685895379836438) - actor/ppo_kl:np.float64(0.0001438019309926153) - actor/pg_clipfrac_lower:np.float64(8.778089977568015e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(88144.51772208884) - self_distillation/token_reweight_w_std:np.float64(1184668.7561488296) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000381489051506) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04390312758914661) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3155992925167084) - perf/mfu/actor:np.float64(0.04083755030532129) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.75739669799805) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/reward/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10212696045932126) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.817) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04379924098110083) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6377375000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0775045006079097) - val-aux/sciknoweval/score/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/score/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10212696045932126) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.817) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04379924098110083) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6377375000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0775045006079097) - val-core/sciknoweval/acc/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/acc/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10212696045932126) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.817) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04379924098110083) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6377375000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0775045006079097) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.007159754662762969) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.002242736757610738) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9955625000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.009417988829529579) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9975125) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.007412161208789384) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99175) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01156057508733171) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99915) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004201813298434218) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9865625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.012175391485948633) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999775) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0021268964117280294) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9809249999999998) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.010081889419878812) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.000683625445693766) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012774050701409578 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012774050701409578 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.84765625 - response_length/max:1465.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.84765625 - response_length_non_aborted/max:1465.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.011104702949524e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0925841629505157) - timing_s/agent_loop/generate_sequences/max:np.float64(10.3709436673671) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332557140849531) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.3709436673671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1465 - timing_s/gen:16.32388431020081 - timing_s/reward:0.06206193566322327 - timing_s/old_log_prob:2.4646734315901995 - timing_s/adv:0.5586496517062187 - timing_s/update_actor:15.928700476884842 - timing_s/step:35.42581527121365 - timing_s/testing:84.386498471722 - timing_s/save_checkpoint:6.650143900886178 - timing_s/stop_profile:0.00012451224029064178 - timing_per_token_ms/update_actor:0.08838230254895182 - timing_per_token_ms/adv:0.0030997345080106464 - timing_per_token_ms/gen:0.14697595381263953 - perf/total_num_tokens:180225 - perf/time_per_step:35.42581527121365 - perf/throughput:635.9239675228005 (TaskRunner pid=2074004) Training Progress: 60%|██████ | 60/100 [51:56<43:41, 65.54s/it] (TaskRunner pid=2074004) step:61 - global_seqlen/min:15258 - global_seqlen/max:27550 - global_seqlen/minmax_diff:12292 - global_seqlen/balanced_min:21784 - global_seqlen/balanced_max:21817 - global_seqlen/mean:21809.5 - actor/entropy:0.11104559153318405 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41980162262916565 - training/rollout_probs_diff_mean:0.0026704224292188883 - training/rollout_probs_diff_std:0.01082498300820589 - training/rollout_actor_probs_pearson_corr:0.9980793595314026 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999408721923828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.5849687012378126e-05 - rollout_corr/rollout_is_max:1.967535376548767 - rollout_corr/rollout_is_min:0.2217474728822708 - rollout_corr/rollout_is_std:0.02934972383081913 - rollout_corr/rollout_is_eff_sample_size:0.9991390970476317 - rollout_corr/rollout_is_seq_mean:1.0000622272491455 - rollout_corr/rollout_is_seq_std:0.002196152927353978 - rollout_corr/rollout_is_seq_max:1.0167111158370972 - rollout_corr/rollout_is_seq_min:0.992173433303833 - rollout_corr/rollout_is_seq_max_deviation:0.016711115837097168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1188437603414059) - rollout_corr/training_log_ppl:np.float64(0.10840739371997188) - rollout_corr/kl:np.float64(0.0004275032880443064) - rollout_corr/k3_kl:np.float64(0.0005405173702293098) - rollout_corr/rollout_ppl:np.float64(1.1183525188826025) - rollout_corr/rollout_log_ppl:np.float64(0.10797988958074711) - rollout_corr/log_ppl_diff:np.float64(0.0004275041392247658) - rollout_corr/log_ppl_abs_diff:np.float64(0.001455118432204472) - rollout_corr/log_ppl_diff_max:np.float64(0.01237882673740387) - rollout_corr/log_ppl_diff_min:np.float64(-0.009973615407943726) - rollout_corr/ppl_ratio:np.float64(1.0004298684652895) - rollout_corr/chi2_token:np.float64(0.001337600639089942) - rollout_corr/chi2_seq:np.float64(0.26282138866372406) - actor/pg_loss:np.float64(0.002470805891789496) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000613119240369997) - actor/ppo_kl:np.float64(-8.526104423367542e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(115092.67977330904) - self_distillation/token_reweight_w_std:np.float64(2121252.4452306055) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004522716626525) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04705037997337058) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.32100969552993774) - perf/mfu/actor:np.float64(0.03935632243399488) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.00535583496094) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0035281332675367594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0035281332675367594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:425.984375 - response_length/max:1622.0 - response_length/min:47.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.984375 - response_length_non_aborted/max:1622.0 - response_length_non_aborted/min:47.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013445131480693817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5521563682705164) - timing_s/agent_loop/generate_sequences/max:np.float64(10.833473060280085) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.15038654532691) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.833473060280085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1622 - timing_s/gen:16.92672893963754 - timing_s/reward:0.05643085576593876 - timing_s/old_log_prob:2.6533840224146843 - timing_s/adv:0.6161332204937935 - timing_s/update_actor:15.688221232965589 - timing_s/step:36.045780032873154 - timing_s/stop_profile:0.00011107698082923889 - timing_per_token_ms/update_actor:0.08991621330707712 - timing_per_token_ms/adv:0.003531335086165395 - timing_per_token_ms/gen:0.15521704269190426 - perf/total_num_tokens:174476 - perf/time_per_step:36.045780032873154 - perf/throughput:605.0500219473707 (TaskRunner pid=2074004) Training Progress: 61%|██████ | 61/100 [52:32<36:51, 56.71s/it] (TaskRunner pid=2074004) step:62 - global_seqlen/min:20453 - global_seqlen/max:31926 - global_seqlen/minmax_diff:11473 - global_seqlen/balanced_min:24980 - global_seqlen/balanced_max:25000 - global_seqlen/mean:24994.125 - actor/entropy:0.11476144939661026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8106536269187927 - training/rollout_probs_diff_mean:0.0027153317350894213 - training/rollout_probs_diff_std:0.011159801855683327 - training/rollout_actor_probs_pearson_corr:0.9979556202888489 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9999613165855408 - rollout_corr/rollout_is_ratio_fraction_high:7.923365046735853e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.338692037388682e-05 - rollout_corr/rollout_is_max:2.658514976501465 - rollout_corr/rollout_is_min:0.1538795679807663 - rollout_corr/rollout_is_std:0.030247831717133522 - rollout_corr/rollout_is_eff_sample_size:0.9990857860453506 - rollout_corr/rollout_is_seq_mean:0.9999266862869263 - rollout_corr/rollout_is_seq_std:0.0018450476927682757 - rollout_corr/rollout_is_seq_max:1.011099934577942 - rollout_corr/rollout_is_seq_min:0.9912953972816467 - rollout_corr/rollout_is_seq_max_deviation:0.011099934577941895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1163019719533622) - rollout_corr/training_log_ppl:np.float64(0.10734543754006154) - rollout_corr/kl:np.float64(0.0005882964252315048) - rollout_corr/k3_kl:np.float64(0.0005425430525072272) - rollout_corr/rollout_ppl:np.float64(1.1156547986902297) - rollout_corr/rollout_log_ppl:np.float64(0.1067571406965726) - rollout_corr/log_ppl_diff:np.float64(0.0005882968434889335) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013848803682776634) - rollout_corr/log_ppl_diff_max:np.float64(0.010647937655448914) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409019231796265) - rollout_corr/ppl_ratio:np.float64(1.0005901597905904) - rollout_corr/chi2_token:np.float64(0.0009822722058743238) - rollout_corr/chi2_seq:np.float64(0.6805941013153642) - actor/pg_loss:np.float64(0.0017424660036340356) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005262740519356157) - actor/ppo_kl:np.float64(4.013105478151147e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(72289.93594874162) - self_distillation/token_reweight_w_std:np.float64(1211116.1148807127) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999488468747586) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04404561784758698) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.22749612480401993) - perf/mfu/actor:np.float64(0.04494797983979118) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.95013809204102) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005711755715310574 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005711755715310574 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:493.00390625 - response_length/max:1745.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:493.00390625 - response_length_non_aborted/max:1745.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.554384112358093e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6613222770392895) - timing_s/agent_loop/generate_sequences/max:np.float64(12.032618755474687) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.850435824046144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.032618755474687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1745 - timing_s/gen:18.026200192049146 - timing_s/reward:0.06495929323136806 - timing_s/old_log_prob:2.460443489253521 - timing_s/adv:0.651430806145072 - timing_s/update_actor:16.151509700343013 - timing_s/step:37.442482167854905 - timing_s/stop_profile:0.00011717900633811951 - timing_per_token_ms/update_actor:0.0807765309864969 - timing_per_token_ms/adv:0.003257919641841193 - timing_per_token_ms/gen:0.14282816750032998 - perf/total_num_tokens:199953 - perf/time_per_step:37.442482167854905 - perf/throughput:667.533869361309 (TaskRunner pid=2074004) Training Progress: 62%|██████▏ | 62/100 [53:09<32:15, 50.94s/it] (TaskRunner pid=2074004) step:63 - global_seqlen/min:16031 - global_seqlen/max:28652 - global_seqlen/minmax_diff:12621 - global_seqlen/balanced_min:20376 - global_seqlen/balanced_max:20475 - global_seqlen/mean:20399.625 - actor/entropy:0.09949057549238205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9106009602546692 - training/rollout_probs_diff_mean:0.002759087597951293 - training/rollout_probs_diff_std:0.011734765022993088 - training/rollout_actor_probs_pearson_corr:0.9973864555358887 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.000015139579773 - rollout_corr/rollout_is_ratio_fraction_high:5.268204404274002e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.2145635234192014e-05 - rollout_corr/rollout_is_max:2.28180193901062 - rollout_corr/rollout_is_min:0.06190122291445732 - rollout_corr/rollout_is_std:0.03141549974679947 - rollout_corr/rollout_is_eff_sample_size:0.9990140393695902 - rollout_corr/rollout_is_seq_mean:0.9999146461486816 - rollout_corr/rollout_is_seq_std:0.001660522655583918 - rollout_corr/rollout_is_seq_max:1.0048866271972656 - rollout_corr/rollout_is_seq_min:0.9946311712265015 - rollout_corr/rollout_is_seq_max_deviation:0.005368828773498535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0948191625066102) - rollout_corr/training_log_ppl:np.float64(0.08782734138367232) - rollout_corr/kl:np.float64(0.000854949058012977) - rollout_corr/k3_kl:np.float64(0.0006357853782912315) - rollout_corr/rollout_ppl:np.float64(1.0938773178495467) - rollout_corr/rollout_log_ppl:np.float64(0.08697239161483594) - rollout_corr/log_ppl_diff:np.float64(0.0008549497688363772) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017329079382761847) - rollout_corr/log_ppl_diff_max:np.float64(0.008672401309013367) - rollout_corr/log_ppl_diff_min:np.float64(-0.0064188651740550995) - rollout_corr/ppl_ratio:np.float64(1.0008579187560827) - rollout_corr/chi2_token:np.float64(0.0008239620365202427) - rollout_corr/chi2_seq:np.float64(0.4398700457531959) - actor/pg_loss:np.float64(0.00171933404635638) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005747673458245117) - actor/ppo_kl:np.float64(0.00029454395632910746) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(141111.829759978) - self_distillation/token_reweight_w_std:np.float64(1970238.3386651743) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000252681784332) - self_distillation/token_reweight_w_clip_frac:np.float64(0.031212968737236224) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.273105651140213) - perf/mfu/actor:np.float64(0.037443201028578815) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.92625427246094) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005000842735171318 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005000842735171318 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.73828125 - response_length/max:1427.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.73828125 - response_length_non_aborted/max:1427.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010591745376586914 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8246997967362404) - timing_s/agent_loop/generate_sequences/max:np.float64(9.28955096192658) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.776446552736161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.28955096192658) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1427 - timing_s/gen:15.555157018825412 - timing_s/reward:0.05943015590310097 - timing_s/old_log_prob:2.3526105862110853 - timing_s/adv:0.5264585241675377 - timing_s/update_actor:15.563456859439611 - timing_s/step:34.14370860904455 - timing_s/stop_profile:0.00012073293328285217 - timing_per_token_ms/update_actor:0.09536607204445922 - timing_per_token_ms/adv:0.0032259080998274336 - timing_per_token_ms/gen:0.1638954895618478 - perf/total_num_tokens:163197 - perf/time_per_step:34.14370860904455 - perf/throughput:597.4636567334168 (TaskRunner pid=2074004) Training Progress: 63%|██████▎ | 63/100 [53:44<28:18, 45.92s/it] (TaskRunner pid=2074004) step:64 - global_seqlen/min:17389 - global_seqlen/max:31739 - global_seqlen/minmax_diff:14350 - global_seqlen/balanced_min:23753 - global_seqlen/balanced_max:30196 - global_seqlen/mean:24563.75 - actor/entropy:0.11078858375549316 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27813637256622314 - training/rollout_probs_diff_mean:0.0025074740406125784 - training/rollout_probs_diff_std:0.010136975906789303 - training/rollout_actor_probs_pearson_corr:0.9982666969299316 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998918771743774 - rollout_corr/rollout_is_ratio_fraction_high:7.794596967869438e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.897298302035779e-05 - rollout_corr/rollout_is_max:2.2562098503112793 - rollout_corr/rollout_is_min:0.38679587841033936 - rollout_corr/rollout_is_std:0.02878168411552906 - rollout_corr/rollout_is_eff_sample_size:0.9991719432652497 - rollout_corr/rollout_is_seq_mean:0.9999525547027588 - rollout_corr/rollout_is_seq_std:0.0016079578781500459 - rollout_corr/rollout_is_seq_max:1.0066720247268677 - rollout_corr/rollout_is_seq_min:0.9929185509681702 - rollout_corr/rollout_is_seq_max_deviation:0.007081449031829834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1173579916357994) - rollout_corr/training_log_ppl:np.float64(0.10709392841090448) - rollout_corr/kl:np.float64(0.0006130551650116445) - rollout_corr/k3_kl:np.float64(0.0006270227015860996) - rollout_corr/rollout_ppl:np.float64(1.1166537292301655) - rollout_corr/rollout_log_ppl:np.float64(0.1064808721675945) - rollout_corr/log_ppl_diff:np.float64(0.000613056243309984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015000321327534039) - rollout_corr/log_ppl_diff_max:np.float64(0.00872485339641571) - rollout_corr/log_ppl_diff_min:np.float64(-0.005040250718593597) - rollout_corr/ppl_ratio:np.float64(1.0006150556728244) - rollout_corr/chi2_token:np.float64(0.0012433729134500027) - rollout_corr/chi2_seq:np.float64(1.6464725509285927) - actor/pg_loss:np.float64(0.0018512200331315398) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047728462982377096) - actor/ppo_kl:np.float64(0.00010965376971583396) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(103190.21550233453) - self_distillation/token_reweight_w_std:np.float64(1658807.6000547835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004303914029151) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04449158857460134) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.2668180838227272) - perf/mfu/actor:np.float64(0.0422181008652012) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.02878189086914) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009036860428750515 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009036860428750515 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:501.1484375 - response_length/max:8192.0 - response_length/min:107.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:501.1484375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013284198939800262 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1968379076570272) - timing_s/agent_loop/generate_sequences/max:np.float64(51.116004686802626) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.855164358850743) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.116004686802626) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:307 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.01408413052559 - timing_s/reward:0.06438678875565529 - timing_s/old_log_prob:2.5394509602338076 - timing_s/adv:0.595608577132225 - timing_s/update_actor:16.805029960349202 - timing_s/step:77.1059439778328 - timing_s/stop_profile:0.00012509338557720184 - timing_per_token_ms/update_actor:0.08551742893669127 - timing_per_token_ms/adv:0.0030309326605883927 - timing_per_token_ms/gen:0.4444017968924937 - perf/total_num_tokens:196510 - perf/time_per_step:77.1059439778328 - perf/throughput:318.57141917699414 (TaskRunner pid=2074004) Training Progress: 64%|██████▍ | 64/100 [55:01<33:10, 55.29s/it] (TaskRunner pid=2074004) step:65 - global_seqlen/min:12881 - global_seqlen/max:28547 - global_seqlen/minmax_diff:15666 - global_seqlen/balanced_min:20761 - global_seqlen/balanced_max:20766 - global_seqlen/mean:20763.75 - actor/entropy:0.10208147764205933 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43991219997406006 - training/rollout_probs_diff_mean:0.00263621611520648 - training/rollout_probs_diff_std:0.01091404352337122 - training/rollout_actor_probs_pearson_corr:0.9977887272834778 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999848008155823 - rollout_corr/rollout_is_ratio_fraction_high:1.0132328498002607e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.052931399201043e-05 - rollout_corr/rollout_is_max:2.1628031730651855 - rollout_corr/rollout_is_min:0.17808698117733002 - rollout_corr/rollout_is_std:0.030277375131845474 - rollout_corr/rollout_is_eff_sample_size:0.9990840011771853 - rollout_corr/rollout_is_seq_mean:0.9999912977218628 - rollout_corr/rollout_is_seq_std:0.0019186957506462932 - rollout_corr/rollout_is_seq_max:1.0081483125686646 - rollout_corr/rollout_is_seq_min:0.9939520359039307 - rollout_corr/rollout_is_seq_max_deviation:0.00814831256866455 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0915365894325078) - rollout_corr/training_log_ppl:np.float64(0.08563249993312638) - rollout_corr/kl:np.float64(0.0005638776894053565) - rollout_corr/k3_kl:np.float64(0.0005349675133992093) - rollout_corr/rollout_ppl:np.float64(1.0908976537175477) - rollout_corr/rollout_log_ppl:np.float64(0.08506862222202471) - rollout_corr/log_ppl_diff:np.float64(0.000563877711101668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015504594302910846) - rollout_corr/log_ppl_diff_max:np.float64(0.007803872227668762) - rollout_corr/log_ppl_diff_min:np.float64(-0.00601176917552948) - rollout_corr/ppl_ratio:np.float64(1.0005660941824317) - rollout_corr/chi2_token:np.float64(0.0010084726382046938) - rollout_corr/chi2_seq:np.float64(0.3784930878318846) - actor/pg_loss:np.float64(0.0010833193082362413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028873655492134276) - actor/ppo_kl:np.float64(7.600492999415565e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(153738.86696914816) - self_distillation/token_reweight_w_std:np.float64(1896279.4650243353) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996677823364735) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026835397191462107) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.22542214393615723) - perf/mfu/actor:np.float64(0.038259118996520076) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.00650787353516) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010626278817653656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010626278817653656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:385.5234375 - response_length/max:1258.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:385.5234375 - response_length_non_aborted/max:1258.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013937614858150482 - timing_s/agent_loop/generate_sequences/min:np.float64(1.065683364868164) - timing_s/agent_loop/generate_sequences/max:np.float64(8.783862510696054) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.777955053730693) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.783862510696054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1258 - timing_s/gen:14.71335643529892 - timing_s/reward:0.06031355261802673 - timing_s/old_log_prob:2.3282906115055084 - timing_s/adv:0.5691004022955894 - timing_s/update_actor:15.625853784382343 - timing_s/step:33.38593070022762 - timing_s/stop_profile:0.00011672265827655792 - timing_per_token_ms/update_actor:0.09406931421577475 - timing_per_token_ms/adv:0.003426045405427665 - timing_per_token_ms/gen:0.1490805564198322 - perf/total_num_tokens:166110 - perf/time_per_step:33.38593070022762 - perf/throughput:621.9311417865741 (TaskRunner pid=2074004) Training Progress: 65%|██████▌ | 65/100 [55:34<28:25, 48.73s/it] (TaskRunner pid=2074004) step:66 - global_seqlen/min:18568 - global_seqlen/max:30571 - global_seqlen/minmax_diff:12003 - global_seqlen/balanced_min:23556 - global_seqlen/balanced_max:23573 - global_seqlen/mean:23566.625 - actor/entropy:0.10450827330350876 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797508478164673 - training/rollout_probs_diff_mean:0.0026548842433840036 - training/rollout_probs_diff_std:0.010791671462357044 - training/rollout_actor_probs_pearson_corr:0.9979157447814941 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000296831130981 - rollout_corr/rollout_is_ratio_fraction_high:2.5662740881671198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.698821718804538e-05 - rollout_corr/rollout_is_max:3.0360376834869385 - rollout_corr/rollout_is_min:0.10401836037635803 - rollout_corr/rollout_is_std:0.030111556872725487 - rollout_corr/rollout_is_eff_sample_size:0.9990941155144449 - rollout_corr/rollout_is_seq_mean:1.000089406967163 - rollout_corr/rollout_is_seq_std:0.0016775390831753612 - rollout_corr/rollout_is_seq_max:1.008592128753662 - rollout_corr/rollout_is_seq_min:0.9943743348121643 - rollout_corr/rollout_is_seq_max_deviation:0.00859212875366211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1033647037111223) - rollout_corr/training_log_ppl:np.float64(0.09674622296006419) - rollout_corr/kl:np.float64(0.0004946426893255307) - rollout_corr/k3_kl:np.float64(0.00056559361085462) - rollout_corr/rollout_ppl:np.float64(1.1028108289465308) - rollout_corr/rollout_log_ppl:np.float64(0.09625157990376465) - rollout_corr/log_ppl_diff:np.float64(0.0004946430562995374) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014105098089203238) - rollout_corr/log_ppl_diff_max:np.float64(0.006463304162025452) - rollout_corr/log_ppl_diff_min:np.float64(-0.007465112954378128) - rollout_corr/ppl_ratio:np.float64(1.000496445223689) - rollout_corr/chi2_token:np.float64(0.001295700902119279) - rollout_corr/chi2_seq:np.float64(1.1554163373075426) - actor/pg_loss:np.float64(0.0020622570300474763) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004437317447809619) - actor/ppo_kl:np.float64(8.988743698523649e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(60744.646061601816) - self_distillation/token_reweight_w_std:np.float64(1164193.8388803264) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999702645232901) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0351556129171513) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.26315246149897575) - perf/mfu/actor:np.float64(0.04286121414241698) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.01708602905273) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013922036159783602 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013922036159783602 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.64453125 - response_length/max:1317.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.64453125 - response_length_non_aborted/max:1317.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.762759923934937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1173687372356653) - timing_s/agent_loop/generate_sequences/max:np.float64(9.68836759403348) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.69582801679644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.68836759403348) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1317 - timing_s/gen:15.383818035945296 - timing_s/reward:0.06612947955727577 - timing_s/old_log_prob:2.3371682837605476 - timing_s/adv:0.568218607455492 - timing_s/update_actor:15.863027606159449 - timing_s/step:34.31379409506917 - timing_s/stop_profile:0.00011265650391578674 - timing_per_token_ms/update_actor:0.08413926265512907 - timing_per_token_ms/adv:0.0030138946892877745 - timing_per_token_ms/gen:0.13159697552583208 - perf/total_num_tokens:188533 - perf/time_per_step:34.31379409506917 - perf/throughput:686.7974125713624 (TaskRunner pid=2074004) Training Progress: 66%|██████▌ | 66/100 [56:09<25:10, 44.42s/it] (TaskRunner pid=2074004) step:67 - global_seqlen/min:13079 - global_seqlen/max:27690 - global_seqlen/minmax_diff:14611 - global_seqlen/balanced_min:20237 - global_seqlen/balanced_max:20247 - global_seqlen/mean:20243.125 - actor/entropy:0.11189419776201248 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6611925363540649 - training/rollout_probs_diff_mean:0.0028133972082287073 - training/rollout_probs_diff_std:0.011245041154325008 - training/rollout_actor_probs_pearson_corr:0.9979284405708313 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.000016450881958 - rollout_corr/rollout_is_ratio_fraction_high:2.1283614842104726e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.385084270732477e-05 - rollout_corr/rollout_is_max:3.327455997467041 - rollout_corr/rollout_is_min:0.31944146752357483 - rollout_corr/rollout_is_std:0.03032655082643032 - rollout_corr/rollout_is_eff_sample_size:0.9990811454013858 - rollout_corr/rollout_is_seq_mean:0.9999868869781494 - rollout_corr/rollout_is_seq_std:0.0020908608566969633 - rollout_corr/rollout_is_seq_max:1.0100783109664917 - rollout_corr/rollout_is_seq_min:0.9929949045181274 - rollout_corr/rollout_is_seq_max_deviation:0.0100783109664917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1118552824482322) - rollout_corr/training_log_ppl:np.float64(0.10242664221368614) - rollout_corr/kl:np.float64(0.0007174603885786723) - rollout_corr/k3_kl:np.float64(0.0005228211956200823) - rollout_corr/rollout_ppl:np.float64(1.111049032304436) - rollout_corr/rollout_log_ppl:np.float64(0.10170918147196062) - rollout_corr/log_ppl_diff:np.float64(0.0007174607417255174) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015918946191959549) - rollout_corr/log_ppl_diff_max:np.float64(0.009511686861515045) - rollout_corr/log_ppl_diff_min:np.float64(-0.008098289370536804) - rollout_corr/ppl_ratio:np.float64(1.000719947507605) - rollout_corr/chi2_token:np.float64(0.0006735411006957293) - rollout_corr/chi2_seq:np.float64(0.3616406440269202) - actor/pg_loss:np.float64(0.0008698338642716408) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023361428748103208) - actor/ppo_kl:np.float64(0.00022379015435802785) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(29018.363461346366) - self_distillation/token_reweight_w_std:np.float64(446767.4180369621) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993730406276882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017900060265674256) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.10916494252160192) - perf/mfu/actor:np.float64(0.03730973843414816) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.93292999267578) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006233438849449158 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:0.006233438849449158 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:367.06640625 - response_length/max:1082.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:367.06640625 - response_length_non_aborted/max:1082.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00020816735923290253 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7952778041362762) - timing_s/agent_loop/generate_sequences/max:np.float64(7.780997171998024) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.720951149596658) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.780997171998024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1082 - timing_s/gen:13.588720228523016 - timing_s/reward:0.05631382390856743 - timing_s/old_log_prob:2.3990508504211903 - timing_s/adv:0.5778900533914566 - timing_s/update_actor:15.670771421864629 - timing_s/step:32.40315775945783 - timing_s/stop_profile:0.00010994821786880493 - timing_per_token_ms/update_actor:0.09676600958266467 - timing_per_token_ms/adv:0.0035684340571889013 - timing_per_token_ms/gen:0.14460854354652083 - perf/total_num_tokens:161945 - perf/time_per_step:32.40315775945783 - perf/throughput:624.7269216868668 (TaskRunner pid=2074004) Training Progress: 67%|██████▋ | 67/100 [56:42<22:32, 40.98s/it] (TaskRunner pid=2074004) step:68 - global_seqlen/min:17605 - global_seqlen/max:31191 - global_seqlen/minmax_diff:13586 - global_seqlen/balanced_min:21836 - global_seqlen/balanced_max:21847 - global_seqlen/mean:21843.875 - actor/entropy:0.09757616370916367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34953224658966064 - training/rollout_probs_diff_mean:0.002497250447049737 - training/rollout_probs_diff_std:0.010688133537769318 - training/rollout_actor_probs_pearson_corr:0.9978483319282532 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000418424606323 - rollout_corr/rollout_is_ratio_fraction_high:2.808383942465298e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.872255961643532e-05 - rollout_corr/rollout_is_max:2.1123147010803223 - rollout_corr/rollout_is_min:0.3512527644634247 - rollout_corr/rollout_is_std:0.030320653691887856 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.0001800060272217 - rollout_corr/rollout_is_seq_std:0.001909724436700344 - rollout_corr/rollout_is_seq_max:1.0095264911651611 - rollout_corr/rollout_is_seq_min:0.9944645166397095 - rollout_corr/rollout_is_seq_max_deviation:0.009526491165161133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0923993606120348) - rollout_corr/training_log_ppl:np.float64(0.08670492109376937) - rollout_corr/kl:np.float64(0.0004997735171485829) - rollout_corr/k3_kl:np.float64(0.0006032356435454744) - rollout_corr/rollout_ppl:np.float64(1.0918437647633255) - rollout_corr/rollout_log_ppl:np.float64(0.08620514698122861) - rollout_corr/log_ppl_diff:np.float64(0.0004997741125407629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014279280803748406) - rollout_corr/log_ppl_diff_max:np.float64(0.0074855126440525055) - rollout_corr/log_ppl_diff_min:np.float64(-0.011258374899625778) - rollout_corr/ppl_ratio:np.float64(1.0005019064992666) - rollout_corr/chi2_token:np.float64(0.0018328602891415358) - rollout_corr/chi2_seq:np.float64(0.43353482661768794) - actor/pg_loss:np.float64(0.0012217597104609013) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019400951759962481) - actor/ppo_kl:np.float64(0.0001972818646009955) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(60185.39716754225) - self_distillation/token_reweight_w_std:np.float64(943506.6543101566) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000661604572088) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01850332456524484) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.25972285121679306) - perf/mfu/actor:np.float64(0.03952048152380881) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.89424896240234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004791805054992437 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.004791805054992437 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:417.27734375 - response_length/max:1557.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.27734375 - response_length_non_aborted/max:1557.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.857063949108124e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8643373139202595) - timing_s/agent_loop/generate_sequences/max:np.float64(10.241223026067019) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.15653961370117) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.241223026067019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1557 - timing_s/gen:16.0516545958817 - timing_s/reward:0.05962240323424339 - timing_s/old_log_prob:2.291767431423068 - timing_s/adv:0.5920419842004776 - timing_s/update_actor:15.795362647622824 - timing_s/step:34.88568533025682 - timing_s/stop_profile:0.00011338107287883759 - timing_per_token_ms/update_actor:0.09038782409040763 - timing_per_token_ms/adv:0.0033879175752955784 - timing_per_token_ms/gen:0.15026403111578687 - perf/total_num_tokens:174751 - perf/time_per_step:34.88568533025682 - perf/throughput:626.1558227452828 (TaskRunner pid=2074004) Training Progress: 68%|██████▊ | 68/100 [57:16<20:53, 39.17s/it] (TaskRunner pid=2074004) step:69 - global_seqlen/min:18936 - global_seqlen/max:25688 - global_seqlen/minmax_diff:6752 - global_seqlen/balanced_min:22204 - global_seqlen/balanced_max:22257 - global_seqlen/mean:22220.75 - actor/entropy:0.10731451213359833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4198373854160309 - training/rollout_probs_diff_mean:0.0025957662146538496 - training/rollout_probs_diff_std:0.010541104711592197 - training/rollout_actor_probs_pearson_corr:0.9981012940406799 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000120997428894 - rollout_corr/rollout_is_ratio_fraction_high:1.8439637642586604e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.687927528517321e-05 - rollout_corr/rollout_is_max:24.342180252075195 - rollout_corr/rollout_is_min:0.34072938561439514 - rollout_corr/rollout_is_std:0.02951376512646675 - rollout_corr/rollout_is_eff_sample_size:0.9991294578066934 - rollout_corr/rollout_is_seq_mean:1.0000665187835693 - rollout_corr/rollout_is_seq_std:0.0023733978159725666 - rollout_corr/rollout_is_seq_max:1.0234099626541138 - rollout_corr/rollout_is_seq_min:0.9945483207702637 - rollout_corr/rollout_is_seq_max_deviation:0.02340996265411377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1120971874333918) - rollout_corr/training_log_ppl:np.float64(0.1024366507299419) - rollout_corr/kl:np.float64(0.00044801794052590216) - rollout_corr/k3_kl:np.float64(0.0006100563592355002) - rollout_corr/rollout_ppl:np.float64(1.111582346726209) - rollout_corr/rollout_log_ppl:np.float64(0.10198863269033609) - rollout_corr/log_ppl_diff:np.float64(0.0004480180396058131) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014786184001422953) - rollout_corr/log_ppl_diff_max:np.float64(0.005983397364616394) - rollout_corr/log_ppl_diff_min:np.float64(-0.010212674736976624) - rollout_corr/ppl_ratio:np.float64(1.0004499880596995) - rollout_corr/chi2_token:np.float64(0.0031010701786726713) - rollout_corr/chi2_seq:np.float64(0.385674498276785) - actor/pg_loss:np.float64(0.001082204282283783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022557121974386973) - actor/ppo_kl:np.float64(-3.220176354723492e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(74030.12377202115) - self_distillation/token_reweight_w_std:np.float64(1098970.7254105331) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995012604631484) - self_distillation/token_reweight_w_clip_frac:np.float64(0.028009024827042595) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.1886149998754263) - perf/mfu/actor:np.float64(0.04061615996018782) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03355026245117) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00016365178453270346 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00016365178453270346 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.6796875 - response_length/max:1407.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.6796875 - response_length_non_aborted/max:1407.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001237001270055771 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7061853520572186) - timing_s/agent_loop/generate_sequences/max:np.float64(9.594152119010687) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.148857997002779) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.594152119010687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:1407 - timing_s/gen:15.759770501405 - timing_s/reward:0.05864102765917778 - timing_s/old_log_prob:2.375713935121894 - timing_s/adv:0.5674699656665325 - timing_s/update_actor:15.860491406172514 - timing_s/step:34.719941690564156 - timing_s/stop_profile:0.00012359023094177246 - timing_per_token_ms/update_actor:0.08922117506256828 - timing_per_token_ms/adv:0.0031922300421145353 - timing_per_token_ms/gen:0.14530223028715128 - perf/total_num_tokens:177766 - perf/time_per_step:34.719941690564156 - perf/throughput:639.9996347355311 (TaskRunner pid=2074004) Training Progress: 69%|██████▉ | 69/100 [57:51<19:33, 37.85s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are to find the magnitude of the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2074385) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/generation_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: 123c1826-04a8-41a1-bd46-2674c1e317a5)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/generation_config.json (WorkerDict pid=2074385) Retrying in 1s [Retry 1/5]. (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2074004) step:70 - global_seqlen/min:16730 - global_seqlen/max:27137 - global_seqlen/minmax_diff:10407 - global_seqlen/balanced_min:21643 - global_seqlen/balanced_max:21648 - global_seqlen/mean:21645.5 - actor/entropy:0.11388496309518814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830116271972656 - training/rollout_probs_diff_mean:0.0028158947825431824 - training/rollout_probs_diff_std:0.011242561973631382 - training/rollout_actor_probs_pearson_corr:0.9978851675987244 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0001726150512695 - rollout_corr/rollout_is_ratio_fraction_high:1.9112419977318496e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.556209988659248e-05 - rollout_corr/rollout_is_max:2.3941094875335693 - rollout_corr/rollout_is_min:0.20454047620296478 - rollout_corr/rollout_is_std:0.03099912963807583 - rollout_corr/rollout_is_eff_sample_size:0.9990402144088839 - rollout_corr/rollout_is_seq_mean:1.0003561973571777 - rollout_corr/rollout_is_seq_std:0.0019966252148151398 - rollout_corr/rollout_is_seq_max:1.0088205337524414 - rollout_corr/rollout_is_seq_min:0.9942991137504578 - rollout_corr/rollout_is_seq_max_deviation:0.008820533752441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1065151896327734) - rollout_corr/training_log_ppl:np.float64(0.09889099766587606) - rollout_corr/kl:np.float64(-3.2485000064363945e-05) - rollout_corr/k3_kl:np.float64(0.0005556557533132889) - rollout_corr/rollout_ppl:np.float64(1.106540143955499) - rollout_corr/rollout_log_ppl:np.float64(0.09892348164066789) - rollout_corr/log_ppl_diff:np.float64(-3.2483974791830406e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015090550186869223) - rollout_corr/log_ppl_diff_max:np.float64(0.014128297567367554) - rollout_corr/log_ppl_diff_min:np.float64(-0.008920878171920776) - rollout_corr/ppl_ratio:np.float64(0.999969924101606) - rollout_corr/chi2_token:np.float64(0.0022924926597625017) - rollout_corr/chi2_seq:np.float64(0.9184615898411721) - actor/pg_loss:np.float64(0.00168591586407274) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006731084324655967) - actor/ppo_kl:np.float64(-0.00014355223837636544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(99180.82018345594) - self_distillation/token_reweight_w_std:np.float64(1940147.8677054113) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003361816052347) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04243566015793476) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.27422183752059937) - perf/mfu/actor:np.float64(0.03944531836232677) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.0858383178711) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.64375) - val-aux/sciknoweval/reward/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09131931477917679) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8197624999999998) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07785458149403679) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6700250000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06801004833603) - val-aux/sciknoweval/score/mean@16:np.float64(0.64375) - val-aux/sciknoweval/score/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09131931477917679) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8197624999999998) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07785458149403679) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6700250000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06801004833603) - val-core/sciknoweval/acc/mean@16:np.float64(0.64375) - val-aux/sciknoweval/acc/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09131931477917679) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8197624999999998) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07785458149403679) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6700250000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06801004833603) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.023033514121596016) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9992374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006965874281756439) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9869375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.030132003555088865) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9933) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.022640924934477863) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9765375000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.037335609975106154) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.997975) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.012752016030515461) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9597750000000002) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04184278385125003) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996875000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0044696795477923035) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9392125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03812973334861979) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0003950870157319778) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011775638908147812 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011775638908147812 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.765625 - response_length/max:1282.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.765625 - response_length_non_aborted/max:1282.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017072074115276337 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7840455882251263) - timing_s/agent_loop/generate_sequences/max:np.float64(8.811903426423669) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.002542052578065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.811903426423669) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1198 - timing_s/gen:14.664157785475254 - timing_s/reward:0.06123044900596142 - timing_s/old_log_prob:2.381863748654723 - timing_s/adv:0.5689894556999207 - timing_s/update_actor:15.963874887675047 - timing_s/step:33.727847304195166 - timing_s/testing:91.3584210369736 - timing_s/save_checkpoint:7.738546384498477 - timing_s/stop_profile:0.00014872848987579346 - timing_per_token_ms/update_actor:0.09218934009190737 - timing_per_token_ms/adv:0.003285841489570122 - timing_per_token_ms/gen:0.1401337657722875 - perf/total_num_tokens:173164 - perf/time_per_step:33.727847304195166 - perf/throughput:641.7693902838463 (TaskRunner pid=2074004) Training Progress: 70%|███████ | 70/100 [1:00:04<33:10, 66.36s/it] (TaskRunner pid=2074004) step:71 - global_seqlen/min:17341 - global_seqlen/max:23612 - global_seqlen/minmax_diff:6271 - global_seqlen/balanced_min:20701 - global_seqlen/balanced_max:20715 - global_seqlen/mean:20711.25 - actor/entropy:0.11080419272184372 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.531029462814331 - training/rollout_probs_diff_mean:0.002882769564166665 - training/rollout_probs_diff_std:0.011653199791908264 - training/rollout_actor_probs_pearson_corr:0.9976783990859985 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000351667404175 - rollout_corr/rollout_is_ratio_fraction_high:1.0566133823886048e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.452907059108838e-05 - rollout_corr/rollout_is_max:2.691452741622925 - rollout_corr/rollout_is_min:0.31489646434783936 - rollout_corr/rollout_is_std:0.031141091138124466 - rollout_corr/rollout_is_eff_sample_size:0.9990312909459407 - rollout_corr/rollout_is_seq_mean:1.0000897645950317 - rollout_corr/rollout_is_seq_std:0.0019137562485411763 - rollout_corr/rollout_is_seq_max:1.0093891620635986 - rollout_corr/rollout_is_seq_min:0.9933313727378845 - rollout_corr/rollout_is_seq_max_deviation:0.009389162063598633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1135450787842274) - rollout_corr/training_log_ppl:np.float64(0.1041237109056965) - rollout_corr/kl:np.float64(0.0005952388999048708) - rollout_corr/k3_kl:np.float64(0.0005321949834606876) - rollout_corr/rollout_ppl:np.float64(1.1128493160940707) - rollout_corr/rollout_log_ppl:np.float64(0.10352847058675252) - rollout_corr/log_ppl_diff:np.float64(0.0005952403189439792) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014877674730087165) - rollout_corr/log_ppl_diff_max:np.float64(0.00793486088514328) - rollout_corr/log_ppl_diff_min:np.float64(-0.004785224795341492) - rollout_corr/ppl_ratio:np.float64(1.0005972273647785) - rollout_corr/chi2_token:np.float64(0.0009104576893150806) - rollout_corr/chi2_seq:np.float64(0.7369661091361195) - actor/pg_loss:np.float64(0.0014374544844031334) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003361999638400448) - actor/ppo_kl:np.float64(0.00016070706497472997) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(28669.062551695388) - self_distillation/token_reweight_w_std:np.float64(576396.9004883079) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999415366910398) - self_distillation/token_reweight_w_clip_frac:np.float64(0.028172490041470155) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.14137845113873482) - perf/mfu/actor:np.float64(0.037758217715448655) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2748680114746) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003997960593551397 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003997960593551397 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:369.6953125 - response_length/max:1046.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.6953125 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014478527009487152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.12398418225348) - timing_s/agent_loop/generate_sequences/max:np.float64(7.80044724419713) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.838224976650963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.80044724419713) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:13.64636124484241 - timing_s/reward:0.05765148997306824 - timing_s/old_log_prob:2.378165924921632 - timing_s/adv:0.5682110711932182 - timing_s/update_actor:15.703643430024385 - timing_s/step:32.44597504287958 - timing_s/stop_profile:0.00010780245065689087 - timing_per_token_ms/update_actor:0.09477725529618194 - timing_per_token_ms/adv:0.0034293624913586712 - timing_per_token_ms/gen:0.14418927373515364 - perf/total_num_tokens:165690 - perf/time_per_step:32.44597504287958 - perf/throughput:638.3303313470673 (TaskRunner pid=2074004) Training Progress: 71%|███████ | 71/100 [1:00:37<27:09, 56.20s/it] (TaskRunner pid=2074004) step:72 - global_seqlen/min:18266 - global_seqlen/max:27074 - global_seqlen/minmax_diff:8808 - global_seqlen/balanced_min:22610 - global_seqlen/balanced_max:22942 - global_seqlen/mean:22655.75 - actor/entropy:0.09876604378223419 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5981020331382751 - training/rollout_probs_diff_mean:0.0024195632431656122 - training/rollout_probs_diff_std:0.010534233413636684 - training/rollout_actor_probs_pearson_corr:0.9980013966560364 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000324249267578 - rollout_corr/rollout_is_ratio_fraction_high:2.6222400265396573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.370399983599782e-05 - rollout_corr/rollout_is_max:2.0842576026916504 - rollout_corr/rollout_is_min:0.0317724235355854 - rollout_corr/rollout_is_std:0.02847144566476345 - rollout_corr/rollout_is_eff_sample_size:0.9991900334087397 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0016953593585640192 - rollout_corr/rollout_is_seq_max:1.010093092918396 - rollout_corr/rollout_is_seq_min:0.9925937652587891 - rollout_corr/rollout_is_seq_max_deviation:0.010093092918395996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1005978477187455) - rollout_corr/training_log_ppl:np.float64(0.09388212007979746) - rollout_corr/kl:np.float64(0.000452447483150209) - rollout_corr/k3_kl:np.float64(0.0004727034649363304) - rollout_corr/rollout_ppl:np.float64(1.1000874377787113) - rollout_corr/rollout_log_ppl:np.float64(0.09342967234624666) - rollout_corr/log_ppl_diff:np.float64(0.00045244773355079815) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013087687766528688) - rollout_corr/log_ppl_diff_max:np.float64(0.008923083543777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.006535425782203674) - rollout_corr/ppl_ratio:np.float64(1.0004541566595435) - rollout_corr/chi2_token:np.float64(0.0009954054839909077) - rollout_corr/chi2_seq:np.float64(0.5610992929432541) - actor/pg_loss:np.float64(0.0011489454191178083) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003143757853649731) - actor/ppo_kl:np.float64(5.7555564531242e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(69880.04204755742) - self_distillation/token_reweight_w_std:np.float64(1271668.7369876243) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998061931692064) - self_distillation/token_reweight_w_clip_frac:np.float64(0.025518682843539864) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.18177369236946106) - perf/mfu/actor:np.float64(0.04106344764567454) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.01401233673096) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008316871710121632 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008316871710121632 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:446.8984375 - response_length/max:1865.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:446.8984375 - response_length_non_aborted/max:1865.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010162405669689178 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4154884051531553) - timing_s/agent_loop/generate_sequences/max:np.float64(12.022635696455836) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.678455237481103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.022635696455836) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1865 - timing_s/gen:17.75904706120491 - timing_s/reward:0.0738532543182373 - timing_s/old_log_prob:2.3650817554444075 - timing_s/adv:0.630233459174633 - timing_s/update_actor:15.826980471611023 - timing_s/step:36.74509644508362 - timing_s/stop_profile:0.0001101028174161911 - timing_per_token_ms/update_actor:0.08732319870017007 - timing_per_token_ms/adv:0.0034772268583838153 - timing_per_token_ms/gen:0.15522828401661548 - perf/total_num_tokens:181246 - perf/time_per_step:36.74509644508362 - perf/throughput:616.5652615406666 (TaskRunner pid=2074004) Training Progress: 72%|███████▏ | 72/100 [1:01:13<23:30, 50.38s/it] (TaskRunner pid=2074004) step:73 - global_seqlen/min:16861 - global_seqlen/max:32908 - global_seqlen/minmax_diff:16047 - global_seqlen/balanced_min:26274 - global_seqlen/balanced_max:26303 - global_seqlen/mean:26293.375 - actor/entropy:0.11385305970907211 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5037192106246948 - training/rollout_probs_diff_mean:0.0026417120825499296 - training/rollout_probs_diff_std:0.011010304093360901 - training/rollout_actor_probs_pearson_corr:0.9979777932167053 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999186396598816 - rollout_corr/rollout_is_ratio_fraction_high:2.1509231373784132e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.603692549513653e-05 - rollout_corr/rollout_is_max:2.2957048416137695 - rollout_corr/rollout_is_min:0.3969012498855591 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990945914883025 - rollout_corr/rollout_is_seq_mean:0.9999231696128845 - rollout_corr/rollout_is_seq_std:0.0021570948883891106 - rollout_corr/rollout_is_seq_max:1.0173187255859375 - rollout_corr/rollout_is_seq_min:0.9907941818237305 - rollout_corr/rollout_is_seq_max_deviation:0.0173187255859375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.119977319613099) - rollout_corr/training_log_ppl:np.float64(0.11065010864695068) - rollout_corr/kl:np.float64(0.0006033849298385796) - rollout_corr/k3_kl:np.float64(0.0006128351814460586) - rollout_corr/rollout_ppl:np.float64(1.1192855625413358) - rollout_corr/rollout_log_ppl:np.float64(0.11004672372655477) - rollout_corr/log_ppl_diff:np.float64(0.0006033849203959107) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016537990886718035) - rollout_corr/log_ppl_diff_max:np.float64(0.010391712188720703) - rollout_corr/log_ppl_diff_min:np.float64(-0.010457545518875122) - rollout_corr/ppl_ratio:np.float64(1.0006065221969038) - rollout_corr/chi2_token:np.float64(0.0012946291826665401) - rollout_corr/chi2_seq:np.float64(1.4108337932266295) - actor/pg_loss:np.float64(0.0022633997723460197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048230918173430837) - actor/ppo_kl:np.float64(-3.4695178307231345e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(100532.0445677184) - self_distillation/token_reweight_w_std:np.float64(1938239.1147645125) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999280977062881) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04582360343192704) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.24655815586447716) - perf/mfu/actor:np.float64(0.04701854823971872) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03591918945312) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002752285450696945 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002752285450696945 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:544.82421875 - response_length/max:1831.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:544.82421875 - response_length_non_aborted/max:1831.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011086836457252502 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8580625150352716) - timing_s/agent_loop/generate_sequences/max:np.float64(13.18264446593821) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.298752498922113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.18264446593821) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1831 - timing_s/gen:19.2261294554919 - timing_s/reward:0.06619958020746708 - timing_s/old_log_prob:2.412239581346512 - timing_s/adv:0.6206546351313591 - timing_s/update_actor:16.13218855857849 - timing_s/step:38.54634447954595 - timing_s/stop_profile:0.0001186523586511612 - timing_per_token_ms/update_actor:0.07669321910261849 - timing_per_token_ms/adv:0.002950622709767 - timing_per_token_ms/gen:0.13784642018635526 - perf/total_num_tokens:210347 - perf/time_per_step:38.54634447954595 - perf/throughput:682.1236969423181 (TaskRunner pid=2074004) Training Progress: 73%|███████▎ | 73/100 [1:01:52<21:04, 46.84s/it] (TaskRunner pid=2074004) step:74 - global_seqlen/min:15082 - global_seqlen/max:27595 - global_seqlen/minmax_diff:12513 - global_seqlen/balanced_min:20956 - global_seqlen/balanced_max:21007 - global_seqlen/mean:20963.375 - actor/entropy:0.1221182718873024 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4268117845058441 - training/rollout_probs_diff_mean:0.0029213139787316322 - training/rollout_probs_diff_std:0.011437395587563515 - training/rollout_actor_probs_pearson_corr:0.9979726672172546 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9998773336410522 - rollout_corr/rollout_is_ratio_fraction_high:9.682135896582622e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.682135714683682e-05 - rollout_corr/rollout_is_max:2.0322699546813965 - rollout_corr/rollout_is_min:0.3633837103843689 - rollout_corr/rollout_is_std:0.03148183599114418 - rollout_corr/rollout_is_eff_sample_size:0.9990097563119252 - rollout_corr/rollout_is_seq_mean:1.0000077486038208 - rollout_corr/rollout_is_seq_std:0.0027337686624377966 - rollout_corr/rollout_is_seq_max:1.0147820711135864 - rollout_corr/rollout_is_seq_min:0.9881354570388794 - rollout_corr/rollout_is_seq_max_deviation:0.014782071113586426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1157428310252726) - rollout_corr/training_log_ppl:np.float64(0.10688100929837674) - rollout_corr/kl:np.float64(0.0004768247579676199) - rollout_corr/k3_kl:np.float64(0.0006016286499246348) - rollout_corr/rollout_ppl:np.float64(1.1151970326900482) - rollout_corr/rollout_log_ppl:np.float64(0.10640418373441207) - rollout_corr/log_ppl_diff:np.float64(0.0004768255639646668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018265418002556544) - rollout_corr/log_ppl_diff_max:np.float64(0.014855928719043732) - rollout_corr/log_ppl_diff_min:np.float64(-0.011131703853607178) - rollout_corr/ppl_ratio:np.float64(1.0004803847987205) - rollout_corr/chi2_token:np.float64(0.0014686237554997206) - rollout_corr/chi2_seq:np.float64(0.33443050901405513) - actor/pg_loss:np.float64(0.0020010630832985044) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046587054316660215) - actor/ppo_kl:np.float64(-7.767568073546727e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(61973.32169602462) - self_distillation/token_reweight_w_std:np.float64(1112938.32659155) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995205644518137) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03652457233692985) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.23524155095219612) - perf/mfu/actor:np.float64(0.03752256342730821) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.06950759887695) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041500055231153965 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0041500055231153965 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:403.44921875 - response_length/max:1678.0 - response_length/min:48.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.44921875 - response_length_non_aborted/max:1678.0 - response_length_non_aborted/min:48.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013217143714427948 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5993990004062653) - timing_s/agent_loop/generate_sequences/max:np.float64(11.049694247543812) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.951693204406183) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.049694247543812) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1678 - timing_s/gen:16.895056227222085 - timing_s/reward:0.058270735666155815 - timing_s/old_log_prob:2.356802634894848 - timing_s/adv:0.6194801684468985 - timing_s/update_actor:16.044949045404792 - timing_s/step:36.06812138669193 - timing_s/stop_profile:0.00010689161717891693 - timing_per_token_ms/update_actor:0.09567250648693729 - timing_per_token_ms/adv:0.003693824160272967 - timing_per_token_ms/gen:0.16358022353361235 - perf/total_num_tokens:167707 - perf/time_per_step:36.06812138669193 - perf/throughput:581.2161597009282 (TaskRunner pid=2074004) Training Progress: 74%|███████▍ | 74/100 [1:02:28<18:54, 43.62s/it] (TaskRunner pid=2074004) step:75 - global_seqlen/min:18314 - global_seqlen/max:24550 - global_seqlen/minmax_diff:6236 - global_seqlen/balanced_min:20293 - global_seqlen/balanced_max:27015 - global_seqlen/mean:21133.625 - actor/entropy:0.07867328822612762 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417161464691162 - training/rollout_probs_diff_mean:0.002312364522367716 - training/rollout_probs_diff_std:0.010702241212129593 - training/rollout_actor_probs_pearson_corr:0.9973950982093811 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999236464500427 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.896926136803813e-05 - rollout_corr/rollout_is_max:1.7065232992172241 - rollout_corr/rollout_is_min:0.09278460592031479 - rollout_corr/rollout_is_std:0.027797745540738106 - rollout_corr/rollout_is_eff_sample_size:0.9992276439614955 - rollout_corr/rollout_is_seq_mean:0.999890923500061 - rollout_corr/rollout_is_seq_std:0.00215591280721128 - rollout_corr/rollout_is_seq_max:1.0128012895584106 - rollout_corr/rollout_is_seq_min:0.9893057942390442 - rollout_corr/rollout_is_seq_max_deviation:0.012801289558410645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.086495197378099) - rollout_corr/training_log_ppl:np.float64(0.0809084869997605) - rollout_corr/kl:np.float64(0.0006969666031730881) - rollout_corr/k3_kl:np.float64(0.000509651741253947) - rollout_corr/rollout_ppl:np.float64(1.085711132735014) - rollout_corr/rollout_log_ppl:np.float64(0.08021151977845875) - rollout_corr/log_ppl_diff:np.float64(0.0006969672213017475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016032274397730362) - rollout_corr/log_ppl_diff_max:np.float64(0.012572050094604492) - rollout_corr/log_ppl_diff_min:np.float64(-0.01109679788351059) - rollout_corr/ppl_ratio:np.float64(1.0006995955482125) - rollout_corr/chi2_token:np.float64(0.0006079066079109907) - rollout_corr/chi2_seq:np.float64(0.20879352744668722) - actor/pg_loss:np.float64(0.0010695301461964846) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000338989874762774) - actor/ppo_kl:np.float64(0.00011593784117280848) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_w_mean:np.float64(98768.57021646714) - self_distillation/token_reweight_w_std:np.float64(1371672.2499703786) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993643839843571) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02514167891058605) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.19157380517572165) - perf/mfu/actor:np.float64(0.0367450268799295) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.9773292541504) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.07149641960859299 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.07149641960859299 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:400.95703125 - response_length/max:8192.0 - response_length/min:76.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:400.95703125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013606809079647064 - timing_s/agent_loop/generate_sequences/min:np.float64(0.973297132179141) - timing_s/agent_loop/generate_sequences/max:np.float64(50.69700602442026) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.94060600863304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.69700602442026) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:261 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.43104676157236 - timing_s/reward:0.05962674133479595 - timing_s/old_log_prob:2.578733177855611 - timing_s/adv:0.56367601826787 - timing_s/update_actor:16.684449028223753 - timing_s/step:76.40252554044127 - timing_s/stop_profile:0.00010974332690238953 - timing_per_token_ms/update_actor:0.09868425925642048 - timing_per_token_ms/adv:0.003333999835971526 - timing_per_token_ms/gen:0.5497690755669771 - perf/total_num_tokens:169069 - perf/time_per_step:76.40252554044127 - perf/throughput:276.6089844610383 (TaskRunner pid=2074004) Training Progress: 75%|███████▌ | 75/100 [1:03:45<22:17, 53.48s/it] (TaskRunner pid=2074004) step:76 - global_seqlen/min:19671 - global_seqlen/max:26890 - global_seqlen/minmax_diff:7219 - global_seqlen/balanced_min:23504 - global_seqlen/balanced_max:23513 - global_seqlen/mean:23508.125 - actor/entropy:0.11660213023424149 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5008273124694824 - training/rollout_probs_diff_mean:0.00285673257894814 - training/rollout_probs_diff_std:0.011434005573391914 - training/rollout_actor_probs_pearson_corr:0.9978346228599548 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998456835746765 - rollout_corr/rollout_is_ratio_fraction_high:1.7103226127801463e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010261935676680878 - rollout_corr/rollout_is_max:3.2767059803009033 - rollout_corr/rollout_is_min:0.304049551486969 - rollout_corr/rollout_is_std:0.03136803209781647 - rollout_corr/rollout_is_eff_sample_size:0.9990166568117982 - rollout_corr/rollout_is_seq_mean:0.9999971985816956 - rollout_corr/rollout_is_seq_std:0.002045826055109501 - rollout_corr/rollout_is_seq_max:1.0144394636154175 - rollout_corr/rollout_is_seq_min:0.9931577444076538 - rollout_corr/rollout_is_seq_max_deviation:0.01443946361541748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.117191367316991) - rollout_corr/training_log_ppl:np.float64(0.108617360041535) - rollout_corr/kl:np.float64(0.0005786410445907109) - rollout_corr/k3_kl:np.float64(0.0006542783519307704) - rollout_corr/rollout_ppl:np.float64(1.1165225864388049) - rollout_corr/rollout_log_ppl:np.float64(0.10803871830285061) - rollout_corr/log_ppl_diff:np.float64(0.0005786417386843823) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014457525903708301) - rollout_corr/log_ppl_diff_max:np.float64(0.007953882217407227) - rollout_corr/log_ppl_diff_min:np.float64(-0.012358836829662323) - rollout_corr/ppl_ratio:np.float64(1.0005807224661112) - rollout_corr/chi2_token:np.float64(0.0018206238746643066) - rollout_corr/chi2_seq:np.float64(0.1669981717132032) - actor/pg_loss:np.float64(0.0019441124750301242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005475720254253247) - actor/ppo_kl:np.float64(3.105731396224609e-05) - actor/pg_clipfrac_lower:np.float64(1.1507097951835021e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(69920.34874183638) - self_distillation/token_reweight_w_std:np.float64(1212741.120844526) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996711185667664) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04925982659915462) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2786094695329666) - perf/mfu/actor:np.float64(0.04275767300860228) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03911590576172) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007215423509478569 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007215423509478569 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.78515625 - response_length/max:1205.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.78515625 - response_length_non_aborted/max:1205.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011574104428291321 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9071431942284107) - timing_s/agent_loop/generate_sequences/max:np.float64(9.096599321812391) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.676992036926094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.096599321812391) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1205 - timing_s/gen:15.07368259690702 - timing_s/reward:0.08101079240441322 - timing_s/old_log_prob:2.4621802624315023 - timing_s/adv:0.5551798567175865 - timing_s/update_actor:15.884726317599416 - timing_s/step:34.14583137631416 - timing_s/stop_profile:0.00012441538274288177 - timing_per_token_ms/update_actor:0.08446402210724704 - timing_per_token_ms/adv:0.0029520636839262303 - timing_per_token_ms/gen:0.1289043040005047 - perf/total_num_tokens:188065 - perf/time_per_step:34.14583137631416 - perf/throughput:688.4625165784311 (TaskRunner pid=2074004) Training Progress: 76%|███████▌ | 76/100 [1:04:19<19:04, 47.70s/it] (TaskRunner pid=2074004) step:77 - global_seqlen/min:15412 - global_seqlen/max:29304 - global_seqlen/minmax_diff:13892 - global_seqlen/balanced_min:21767 - global_seqlen/balanced_max:21798 - global_seqlen/mean:21789.125 - actor/entropy:0.0992889478802681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9991886615753174 - training/rollout_probs_diff_mean:0.00266986689530313 - training/rollout_probs_diff_std:0.011961651965975761 - training/rollout_actor_probs_pearson_corr:0.9973249435424805 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9998555779457092 - rollout_corr/rollout_is_ratio_fraction_high:2.7756959752878174e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.327088289661333e-05 - rollout_corr/rollout_is_max:4.194519519805908 - rollout_corr/rollout_is_min:6.96785764375818e-06 - rollout_corr/rollout_is_std:0.03096642531454563 - rollout_corr/rollout_is_eff_sample_size:0.999041761158671 - rollout_corr/rollout_is_seq_mean:0.9997901320457458 - rollout_corr/rollout_is_seq_std:0.002063008723780513 - rollout_corr/rollout_is_seq_max:1.0076104402542114 - rollout_corr/rollout_is_seq_min:0.991796612739563 - rollout_corr/rollout_is_seq_max_deviation:0.008203387260437012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.100688367150724) - rollout_corr/training_log_ppl:np.float64(0.09442752133691101) - rollout_corr/kl:np.float64(0.0008365866270294564) - rollout_corr/k3_kl:np.float64(0.0005724456864371064) - rollout_corr/rollout_ppl:np.float64(1.0997380977496505) - rollout_corr/rollout_log_ppl:np.float64(0.0935909333784366) - rollout_corr/log_ppl_diff:np.float64(0.0008365879584744107) - rollout_corr/log_ppl_abs_diff:np.float64(0.001732913853629725) - rollout_corr/log_ppl_diff_max:np.float64(0.011707089841365814) - rollout_corr/log_ppl_diff_min:np.float64(-0.0054268017411231995) - rollout_corr/ppl_ratio:np.float64(1.0008395314216614) - rollout_corr/chi2_token:np.float64(0.0006989024113863707) - rollout_corr/chi2_seq:np.float64(0.36190675757825375) - actor/pg_loss:np.float64(0.0014117311220616102) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031758257955516456) - actor/ppo_kl:np.float64(3.489058610828266e-05) - actor/pg_clipfrac_lower:np.float64(1.2245297511981335e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(78938.03803747497) - self_distillation/token_reweight_w_std:np.float64(1405159.017775763) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000561457127333) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02887801337055862) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.1942897979170084) - perf/mfu/actor:np.float64(0.039820481115549144) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.1161994934082) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002417168579995632 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002417168579995632 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:422.19140625 - response_length/max:1559.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:422.19140625 - response_length_non_aborted/max:1559.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013943202793598175 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1664074026048183) - timing_s/agent_loop/generate_sequences/max:np.float64(10.330939007923007) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.180417315277737) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.330939007923007) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:1540 - timing_s/gen:16.243871303275228 - timing_s/reward:0.05987227149307728 - timing_s/old_log_prob:2.3679722510278225 - timing_s/adv:0.5635029003024101 - timing_s/update_actor:15.628741595894098 - timing_s/step:34.95607234351337 - timing_s/stop_profile:0.00011238269507884979 - timing_per_token_ms/update_actor:0.08965907072848324 - timing_per_token_ms/adv:0.0032327072582217626 - timing_per_token_ms/gen:0.15029349564932992 - perf/total_num_tokens:174313 - perf/time_per_step:34.95607234351337 - perf/throughput:623.3287534674445 (TaskRunner pid=2074004) Training Progress: 77%|███████▋ | 77/100 [1:04:54<16:49, 43.89s/it] (TaskRunner pid=2074004) step:78 - global_seqlen/min:14364 - global_seqlen/max:26342 - global_seqlen/minmax_diff:11978 - global_seqlen/balanced_min:20665 - global_seqlen/balanced_max:20678 - global_seqlen/mean:20672.125 - actor/entropy:0.10278760641813278 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30106163024902344 - training/rollout_probs_diff_mean:0.0025821924209594727 - training/rollout_probs_diff_std:0.010455216281116009 - training/rollout_actor_probs_pearson_corr:0.9980041980743408 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:0.9999585151672363 - rollout_corr/rollout_is_ratio_fraction_high:1.0236567504762206e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0946270019048825e-05 - rollout_corr/rollout_is_max:2.61385440826416 - rollout_corr/rollout_is_min:0.18725009262561798 - rollout_corr/rollout_is_std:0.03052050806581974 - rollout_corr/rollout_is_eff_sample_size:0.9990692465112823 - rollout_corr/rollout_is_seq_mean:0.9999092817306519 - rollout_corr/rollout_is_seq_std:0.0021261873189359903 - rollout_corr/rollout_is_seq_max:1.0074591636657715 - rollout_corr/rollout_is_seq_min:0.9891470670700073 - rollout_corr/rollout_is_seq_max_deviation:0.010852932929992676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1040684799663723) - rollout_corr/training_log_ppl:np.float64(0.09624901220013271) - rollout_corr/kl:np.float64(0.0012303921190799372) - rollout_corr/k3_kl:np.float64(0.0009560748116967943) - rollout_corr/rollout_ppl:np.float64(1.1026978376321495) - rollout_corr/rollout_log_ppl:np.float64(0.09501861992612248) - rollout_corr/log_ppl_diff:np.float64(0.001230392274010228) - rollout_corr/log_ppl_abs_diff:np.float64(0.002099611494486453) - rollout_corr/log_ppl_diff_max:np.float64(0.024470895528793335) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052408576011657715) - rollout_corr/ppl_ratio:np.float64(1.0012363716959953) - rollout_corr/chi2_token:np.float64(0.0011034025810658932) - rollout_corr/chi2_seq:np.float64(0.31427349406294525) - actor/pg_loss:np.float64(0.0014356522588059306) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005964356096228585) - actor/ppo_kl:np.float64(0.0006484976534100184) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_w_mean:np.float64(93486.41973545495) - self_distillation/token_reweight_w_std:np.float64(1284691.6428641423) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996571952942759) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02805576975515578) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.22916332073509693) - perf/mfu/actor:np.float64(0.03750177780047932) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.05967712402344) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009613416157662868 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009613416157662868 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:381.59765625 - response_length/max:1424.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.59765625 - response_length_non_aborted/max:1424.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.280653178691864e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.881490908563137) - timing_s/agent_loop/generate_sequences/max:np.float64(9.613876366987824) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7271649866961525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.613876366987824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1424 - timing_s/gen:15.345683528110385 - timing_s/reward:0.0597789641469717 - timing_s/old_log_prob:2.3274049311876297 - timing_s/adv:0.6073618121445179 - timing_s/update_actor:15.772901671007276 - timing_s/step:34.20114932395518 - timing_s/stop_profile:0.00012829527258872986 - timing_per_token_ms/update_actor:0.09537542506519815 - timing_per_token_ms/adv:0.003672589369407583 - timing_per_token_ms/gen:0.15708711859175942 - perf/total_num_tokens:165377 - perf/time_per_step:34.20114932395518 - perf/throughput:604.4277870369937 (TaskRunner pid=2074004) Training Progress: 78%|███████▊ | 78/100 [1:05:28<15:01, 41.00s/it] (TaskRunner pid=2074004) step:79 - global_seqlen/min:17456 - global_seqlen/max:34104 - global_seqlen/minmax_diff:16648 - global_seqlen/balanced_min:23010 - global_seqlen/balanced_max:23387 - global_seqlen/mean:23066.5 - actor/entropy:0.10228408128023148 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4396432042121887 - training/rollout_probs_diff_mean:0.0026395104359835386 - training/rollout_probs_diff_std:0.011051760986447334 - training/rollout_actor_probs_pearson_corr:0.9977607131004333 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.984375 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.984375 - rollout_corr/rollout_is_mean:1.000108003616333 - rollout_corr/rollout_is_ratio_fraction_high:8.729964974918403e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.364982305560261e-05 - rollout_corr/rollout_is_max:2.167071580886841 - rollout_corr/rollout_is_min:0.29309991002082825 - rollout_corr/rollout_is_std:0.030481424182653427 - rollout_corr/rollout_is_eff_sample_size:0.9990721022190584 - rollout_corr/rollout_is_seq_mean:1.0001380443572998 - rollout_corr/rollout_is_seq_std:0.0018053654348477721 - rollout_corr/rollout_is_seq_max:1.0080220699310303 - rollout_corr/rollout_is_seq_min:0.9950286149978638 - rollout_corr/rollout_is_seq_max_deviation:0.008022069931030273 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1028944421559572) - rollout_corr/training_log_ppl:np.float64(0.09561884048889624) - rollout_corr/kl:np.float64(0.000491056393927991) - rollout_corr/k3_kl:np.float64(0.0006740913643881186) - rollout_corr/rollout_ppl:np.float64(1.1023467793129385) - rollout_corr/rollout_log_ppl:np.float64(0.0951277833446511) - rollout_corr/log_ppl_diff:np.float64(0.0004910571442451328) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015971105312928557) - rollout_corr/log_ppl_diff_max:np.float64(0.012345612049102783) - rollout_corr/log_ppl_diff_min:np.float64(-0.006457008421421051) - rollout_corr/ppl_ratio:np.float64(1.00049376511015) - rollout_corr/chi2_token:np.float64(0.0030803862027823925) - rollout_corr/chi2_seq:np.float64(0.8471683100797236) - actor/pg_loss:np.float64(0.0012736148200929165) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005856991260770883) - actor/ppo_kl:np.float64(0.00015690973241966333) - actor/pg_clipfrac_lower:np.float64(1.272394092666218e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(105670.61742275464) - self_distillation/token_reweight_w_std:np.float64(1558575.511290349) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007708300836384) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034760000635287724) - self_distillation/empty_target_batch:np.float64(0.015625) - actor/grad_norm:np.float64(0.2251501902937889) - perf/mfu/actor:np.float64(0.04172759803828769) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.39704513549805) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00025862519396468997 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00025862519396468997 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:447.453125 - response_length/max:1822.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:447.453125 - response_length_non_aborted/max:1822.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.048916399478912e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.363132268190384) - timing_s/agent_loop/generate_sequences/max:np.float64(12.375673668459058) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.632285126703209) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.375673668459058) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1822 - timing_s/gen:18.45864589139819 - timing_s/reward:0.39899917505681515 - timing_s/old_log_prob:2.5533297322690487 - timing_s/adv:0.6657597441226244 - timing_s/update_actor:15.879793219268322 - timing_s/step:37.99234522506595 - timing_s/stop_profile:0.00012224353849887848 - timing_per_token_ms/update_actor:0.08605441451492599 - timing_per_token_ms/adv:0.0036078281497118354 - timing_per_token_ms/gen:0.16114332761286265 - perf/total_num_tokens:184532 - perf/time_per_step:37.99234522506595 - perf/throughput:607.1354601395224 (TaskRunner pid=2074004) Training Progress: 79%|███████▉ | 79/100 [1:06:06<14:02, 40.11s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2074004) step:80 - global_seqlen/min:15487 - global_seqlen/max:29827 - global_seqlen/minmax_diff:14340 - global_seqlen/balanced_min:22537 - global_seqlen/balanced_max:22562 - global_seqlen/mean:22553.375 - actor/entropy:0.09155089408159256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730071544647217 - training/rollout_probs_diff_mean:0.0024328790605068207 - training/rollout_probs_diff_std:0.010556120425462723 - training/rollout_actor_probs_pearson_corr:0.9977543950080872 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000025033950806 - rollout_corr/rollout_is_ratio_fraction_high:3.5952147300122306e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.4940185034647584e-05 - rollout_corr/rollout_is_max:4.466158390045166 - rollout_corr/rollout_is_min:0.25997382402420044 - rollout_corr/rollout_is_std:0.029982615262269974 - rollout_corr/rollout_is_eff_sample_size:0.9991017311505869 - rollout_corr/rollout_is_seq_mean:0.9999626278877258 - rollout_corr/rollout_is_seq_std:0.0018385304138064384 - rollout_corr/rollout_is_seq_max:1.005773901939392 - rollout_corr/rollout_is_seq_min:0.9919787049293518 - rollout_corr/rollout_is_seq_max_deviation:0.008021295070648193 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0906855990178883) - rollout_corr/training_log_ppl:np.float64(0.08496327018656302) - rollout_corr/kl:np.float64(0.0005949745295730224) - rollout_corr/k3_kl:np.float64(0.0005148513625989892) - rollout_corr/rollout_ppl:np.float64(1.0900255064480007) - rollout_corr/rollout_log_ppl:np.float64(0.08436829397760448) - rollout_corr/log_ppl_diff:np.float64(0.000594976208958542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013135308654455002) - rollout_corr/log_ppl_diff_max:np.float64(0.009091086685657501) - rollout_corr/log_ppl_diff_min:np.float64(-0.007834956049919128) - rollout_corr/ppl_ratio:np.float64(1.00059686251916) - rollout_corr/chi2_token:np.float64(0.0010652847122401) - rollout_corr/chi2_seq:np.float64(0.4150397318881005) - actor/pg_loss:np.float64(0.0015692964661866426) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005134487364557572) - actor/ppo_kl:np.float64(7.53170088216848e-05) - actor/pg_clipfrac_lower:np.float64(5.053363565821201e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(107108.59001243045) - self_distillation/token_reweight_w_std:np.float64(1823206.4579460123) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007828855887055) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0354066689760657) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2679561898112297) - perf/mfu/actor:np.float64(0.04098502127101053) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.34692001342773) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/reward/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07668878228681326) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320874999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07047066140853427) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.67935) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05739226575463654) - val-aux/sciknoweval/score/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/score/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07668878228681326) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8320874999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07047066140853427) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.67935) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05739226575463654) - val-core/sciknoweval/acc/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/acc/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07668878228681326) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8320874999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07047066140853427) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.67935) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05739226575463654) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0030257682392245445) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0008816709987291178) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9984999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.00406201920231798) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9993124999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0028497532787945) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9971375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.005252365538497868) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.001615887913810856) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.995125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.006096874199128599) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9923249999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006085382075104241) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00442548468708992 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00442548468708992 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.60546875 - response_length/max:1332.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.60546875 - response_length_non_aborted/max:1332.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.599832653999329e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1593389865010977) - timing_s/agent_loop/generate_sequences/max:np.float64(9.577507866546512) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.48384735804575) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.577507866546512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1332 - timing_s/gen:15.530941212549806 - timing_s/reward:0.05138901062309742 - timing_s/old_log_prob:2.2361939437687397 - timing_s/adv:0.5929265525192022 - timing_s/update_actor:15.585156086832285 - timing_s/step:34.08298168331385 - timing_s/testing:84.83454155549407 - timing_s/save_checkpoint:6.7697754595428705 - timing_s/stop_profile:0.00011528097093105316 - timing_per_token_ms/update_actor:0.08637928961204412 - timing_per_token_ms/adv:0.0032862407096454645 - timing_per_token_ms/gen:0.1395926730650986 - perf/total_num_tokens:180427 - perf/time_per_step:34.08298168331385 - perf/throughput:661.7195411351453 (TaskRunner pid=2074004) Training Progress: 80%|████████ | 80/100 [1:08:12<21:56, 65.81s/it] (TaskRunner pid=2074004) step:81 - global_seqlen/min:17592 - global_seqlen/max:32104 - global_seqlen/minmax_diff:14512 - global_seqlen/balanced_min:22879 - global_seqlen/balanced_max:22902 - global_seqlen/mean:22894.0 - actor/entropy:0.11737882345914841 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.0028525586239993572 - training/rollout_probs_diff_std:0.01126584131270647 - training/rollout_actor_probs_pearson_corr:0.9979852437973022 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.76953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:0.9998797178268433 - rollout_corr/rollout_is_ratio_fraction_high:8.564577001379803e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.2822885006899014e-05 - rollout_corr/rollout_is_max:2.0156476497650146 - rollout_corr/rollout_is_min:0.12281379103660583 - rollout_corr/rollout_is_std:0.03066662885248661 - rollout_corr/rollout_is_eff_sample_size:0.9990602035443582 - rollout_corr/rollout_is_seq_mean:1.000148892402649 - rollout_corr/rollout_is_seq_std:0.002365858294069767 - rollout_corr/rollout_is_seq_max:1.0138297080993652 - rollout_corr/rollout_is_seq_min:0.9926231503486633 - rollout_corr/rollout_is_seq_max_deviation:0.013829708099365234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1165403798222542) - rollout_corr/training_log_ppl:np.float64(0.10833677103437367) - rollout_corr/kl:np.float64(0.0006071828474922825) - rollout_corr/k3_kl:np.float64(0.0006022309770301604) - rollout_corr/rollout_ppl:np.float64(1.1158664338290691) - rollout_corr/rollout_log_ppl:np.float64(0.10772958788948017) - rollout_corr/log_ppl_diff:np.float64(0.0006071831448934972) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018550536478869617) - rollout_corr/log_ppl_diff_max:np.float64(0.009145308285951614) - rollout_corr/log_ppl_diff_min:np.float64(-0.012295559048652649) - rollout_corr/ppl_ratio:np.float64(1.0006107091903687) - rollout_corr/chi2_token:np.float64(0.001226232387125492) - rollout_corr/chi2_seq:np.float64(0.12261000624857843) - actor/pg_loss:np.float64(0.0018157117301598191) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004039403577280609) - actor/ppo_kl:np.float64(0.00021660448263460808) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_w_mean:np.float64(76784.49322666228) - self_distillation/token_reweight_w_std:np.float64(1652886.481738999) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002255805302411) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04571778689569328) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.2025649379938841) - perf/mfu/actor:np.float64(0.04102161940188653) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.92991256713867) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009963815100491047 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009963815100491047 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.09375 - response_length/max:1588.0 - response_length/min:61.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.09375 - response_length_non_aborted/max:1588.0 - response_length_non_aborted/min:61.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.460662603378296e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6343090515583754) - timing_s/agent_loop/generate_sequences/max:np.float64(10.963617943227291) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.317745556931186) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.963617943227291) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1588 - timing_s/gen:16.757680144160986 - timing_s/reward:0.056961143389344215 - timing_s/old_log_prob:2.566490786150098 - timing_s/adv:0.6639319956302643 - timing_s/update_actor:15.944705080240965 - timing_s/step:36.09860094822943 - timing_s/stop_profile:0.00011157244443893433 - timing_per_token_ms/update_actor:0.08705722613043246 - timing_per_token_ms/adv:0.0036250327358165035 - timing_per_token_ms/gen:0.14352244042618179 - perf/total_num_tokens:183152 - perf/time_per_step:36.09860094822943 - perf/throughput:634.2074041271926 (TaskRunner pid=2074004) Training Progress: 81%|████████ | 81/100 [1:08:48<18:01, 56.91s/it] (TaskRunner pid=2074004) step:82 - global_seqlen/min:18726 - global_seqlen/max:28987 - global_seqlen/minmax_diff:10261 - global_seqlen/balanced_min:23632 - global_seqlen/balanced_max:23847 - global_seqlen/mean:23672.375 - actor/entropy:0.10426922142505646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.506376326084137 - training/rollout_probs_diff_mean:0.0025403054896742105 - training/rollout_probs_diff_std:0.010555604472756386 - training/rollout_actor_probs_pearson_corr:0.9980065822601318 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0000488758087158 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.208931204630062e-05 - rollout_corr/rollout_is_max:1.914077877998352 - rollout_corr/rollout_is_min:0.227353036403656 - rollout_corr/rollout_is_std:0.02918272092938423 - rollout_corr/rollout_is_eff_sample_size:0.999149212500466 - rollout_corr/rollout_is_seq_mean:0.9999838471412659 - rollout_corr/rollout_is_seq_std:0.0015772389015182853 - rollout_corr/rollout_is_seq_max:1.0057252645492554 - rollout_corr/rollout_is_seq_min:0.9941803216934204 - rollout_corr/rollout_is_seq_max_deviation:0.00581967830657959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.109215888660401) - rollout_corr/training_log_ppl:np.float64(0.10109948862009333) - rollout_corr/kl:np.float64(0.0005538763568267713) - rollout_corr/k3_kl:np.float64(0.0005117594077042042) - rollout_corr/rollout_ppl:np.float64(1.1085950387641788) - rollout_corr/rollout_log_ppl:np.float64(0.10054561170181842) - rollout_corr/log_ppl_diff:np.float64(0.0005538769182749093) - rollout_corr/log_ppl_abs_diff:np.float64(0.001272055131266825) - rollout_corr/log_ppl_diff_max:np.float64(0.006336919963359833) - rollout_corr/log_ppl_diff_min:np.float64(-0.004927821457386017) - rollout_corr/ppl_ratio:np.float64(1.000555366044864) - rollout_corr/chi2_token:np.float64(0.0009450593497604132) - rollout_corr/chi2_seq:np.float64(1.0984282961580902) - actor/pg_loss:np.float64(0.0019886784721165895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003411252544083254) - actor/ppo_kl:np.float64(9.175821236606652e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(50298.95451163477) - self_distillation/token_reweight_w_std:np.float64(977247.3528398918) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999259575270116) - self_distillation/token_reweight_w_clip_frac:np.float64(0.038464386205305345) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.25309666618704796) - perf/mfu/actor:np.float64(0.04338852253393083) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.42945861816406) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00773917231708765 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00773917231708765 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.04296875 - response_length/max:1880.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:464.04296875 - response_length_non_aborted/max:1880.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001201629638671875 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6977196279913187) - timing_s/agent_loop/generate_sequences/max:np.float64(12.517238644883037) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.792556140542729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.517238644883037) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:349 - timing_s/agent_loop/slowest/response_length:1880 - timing_s/gen:18.328095687553287 - timing_s/reward:0.06305506825447083 - timing_s/old_log_prob:2.362060610204935 - timing_s/adv:0.6524072252213955 - timing_s/update_actor:15.651612892746925 - timing_s/step:37.145222717896104 - timing_s/stop_profile:4.989840090274811e-05 - timing_per_token_ms/update_actor:0.08264703527184601 - timing_per_token_ms/adv:0.0034449818893403992 - timing_per_token_ms/gen:0.1542833931356815 - perf/total_num_tokens:189379 - perf/time_per_step:37.145222717896104 - perf/throughput:637.2925848306987 (TaskRunner pid=2074004) Training Progress: 82%|████████▏ | 82/100 [1:09:25<15:17, 50.99s/it] (TaskRunner pid=2074004) step:83 - global_seqlen/min:16528 - global_seqlen/max:26927 - global_seqlen/minmax_diff:10399 - global_seqlen/balanced_min:21714 - global_seqlen/balanced_max:21743 - global_seqlen/mean:21736.25 - actor/entropy:0.1269112080335617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6248966455459595 - training/rollout_probs_diff_mean:0.0030177931766957045 - training/rollout_probs_diff_std:0.011725279502570629 - training/rollout_actor_probs_pearson_corr:0.9978418350219727 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:3.8849284464959055e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.769856892991811e-05 - rollout_corr/rollout_is_max:2.3085124492645264 - rollout_corr/rollout_is_min:0.010780704207718372 - rollout_corr/rollout_is_std:0.03224320337176323 - rollout_corr/rollout_is_eff_sample_size:0.9989612175582933 - rollout_corr/rollout_is_seq_mean:0.9999208450317383 - rollout_corr/rollout_is_seq_std:0.001950851990841329 - rollout_corr/rollout_is_seq_max:1.0089315176010132 - rollout_corr/rollout_is_seq_min:0.9943766593933105 - rollout_corr/rollout_is_seq_max_deviation:0.008931517601013184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1371404435485601) - rollout_corr/training_log_ppl:np.float64(0.12448573324945755) - rollout_corr/kl:np.float64(0.0008972727727523111) - rollout_corr/k3_kl:np.float64(0.0008475244219425804) - rollout_corr/rollout_ppl:np.float64(1.1360785402357578) - rollout_corr/rollout_log_ppl:np.float64(0.12358846047573024) - rollout_corr/log_ppl_diff:np.float64(0.000897272773727309) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018847366955014877) - rollout_corr/log_ppl_diff_max:np.float64(0.01430445909500122) - rollout_corr/log_ppl_diff_min:np.float64(-0.007731936872005463) - rollout_corr/ppl_ratio:np.float64(1.0009009479545057) - rollout_corr/chi2_token:np.float64(0.001457967096939683) - rollout_corr/chi2_seq:np.float64(0.5747340724337846) - actor/pg_loss:np.float64(0.00172864377964288) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005559734236157965) - actor/ppo_kl:np.float64(0.0002423281556467316) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(34890.77068122127) - self_distillation/token_reweight_w_std:np.float64(689731.6690365797) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998807616066188) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03181662493443582) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.24900896847248077) - perf/mfu/actor:np.float64(0.039740719700471376) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4448585510254) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005997358239255846 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005997358239255846 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:402.1953125 - response_length/max:1439.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:402.1953125 - response_length_non_aborted/max:1439.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.76057893037796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6644078604876995) - timing_s/agent_loop/generate_sequences/max:np.float64(10.04941943101585) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.103168618654308) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.04941943101585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1439 - timing_s/gen:15.86747464723885 - timing_s/reward:0.06120395101606846 - timing_s/old_log_prob:2.3169026523828506 - timing_s/adv:0.5539372637867928 - timing_s/update_actor:15.275738924741745 - timing_s/step:34.1625665910542 - timing_s/stop_profile:3.450922667980194e-05 - timing_per_token_ms/update_actor:0.08784713856312465 - timing_per_token_ms/adv:0.0031855613536534173 - timing_per_token_ms/gen:0.1541100080344093 - perf/total_num_tokens:173890 - perf/time_per_step:34.1625665910542 - perf/throughput:636.2592793508626 (TaskRunner pid=2074004) Training Progress: 83%|████████▎ | 83/100 [1:09:59<13:01, 45.94s/it] (TaskRunner pid=2074004) step:84 - global_seqlen/min:18342 - global_seqlen/max:28767 - global_seqlen/minmax_diff:10425 - global_seqlen/balanced_min:22295 - global_seqlen/balanced_max:22560 - global_seqlen/mean:22373.75 - actor/entropy:0.11953727900981903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016381561756134 - training/rollout_probs_diff_mean:0.0028403468895703554 - training/rollout_probs_diff_std:0.010894829407334328 - training/rollout_actor_probs_pearson_corr:0.9980471134185791 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000654458999634 - rollout_corr/rollout_is_ratio_fraction_high:8.987794899439905e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.6963383788825013e-05 - rollout_corr/rollout_is_max:2.1378886699676514 - rollout_corr/rollout_is_min:0.1871485859155655 - rollout_corr/rollout_is_std:0.030487289652228355 - rollout_corr/rollout_is_eff_sample_size:0.9990715072785922 - rollout_corr/rollout_is_seq_mean:1.0000371932983398 - rollout_corr/rollout_is_seq_std:0.0019689607433974743 - rollout_corr/rollout_is_seq_max:1.0071165561676025 - rollout_corr/rollout_is_seq_min:0.9907252788543701 - rollout_corr/rollout_is_seq_max_deviation:0.009274721145629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1271462766453624) - rollout_corr/training_log_ppl:np.float64(0.11585301743616583) - rollout_corr/kl:np.float64(0.0005694015161132882) - rollout_corr/k3_kl:np.float64(0.0006418242667365348) - rollout_corr/rollout_ppl:np.float64(1.1264640409499407) - rollout_corr/rollout_log_ppl:np.float64(0.11528361437740386) - rollout_corr/log_ppl_diff:np.float64(0.0005694030587619636) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015966345054039266) - rollout_corr/log_ppl_diff_max:np.float64(0.010501980781555176) - rollout_corr/log_ppl_diff_min:np.float64(-0.006051577627658844) - rollout_corr/ppl_ratio:np.float64(1.0005718814209104) - rollout_corr/chi2_token:np.float64(0.0016377929132431746) - rollout_corr/chi2_seq:np.float64(0.6639628079719841) - actor/pg_loss:np.float64(0.002379689714871347) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000606918740686524) - actor/ppo_kl:np.float64(9.888500281718393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(92119.91488653235) - self_distillation/token_reweight_w_std:np.float64(1798317.0580325122) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002044991124421) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05570600633654976) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.29781750962138176) - perf/mfu/actor:np.float64(0.040360571878840845) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.56901168823242) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005010695313103497 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005010695313103497 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.6171875 - response_length/max:2171.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.6171875 - response_length_non_aborted/max:2171.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.546158015727997e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7891156543046236) - timing_s/agent_loop/generate_sequences/max:np.float64(13.379426375031471) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.27586570179119) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.379426375031471) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:2171 - timing_s/gen:19.16517908498645 - timing_s/reward:0.06223759055137634 - timing_s/old_log_prob:2.38674658536911 - timing_s/adv:0.6694579813629389 - timing_s/update_actor:15.630993586033583 - timing_s/step:37.9998825173825 - timing_s/stop_profile:3.123842179775238e-05 - timing_per_token_ms/update_actor:0.08732886522170838 - timing_per_token_ms/adv:0.003740197672288613 - timing_per_token_ms/gen:0.17225269260831594 - perf/total_num_tokens:178990 - perf/time_per_step:37.9998825173825 - perf/throughput:588.7847150518281 (TaskRunner pid=2074004) Training Progress: 84%|████████▍ | 84/100 [1:10:37<11:37, 43.57s/it] (TaskRunner pid=2074004) step:85 - global_seqlen/min:13444 - global_seqlen/max:29698 - global_seqlen/minmax_diff:16254 - global_seqlen/balanced_min:21579 - global_seqlen/balanced_max:21625 - global_seqlen/mean:21607.0 - actor/entropy:0.1099928542971611 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7565039396286011 - training/rollout_probs_diff_mean:0.002620653947815299 - training/rollout_probs_diff_std:0.011461569927632809 - training/rollout_actor_probs_pearson_corr:0.9977118968963623 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.72265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9998971819877625 - rollout_corr/rollout_is_ratio_fraction_high:9.451795449422207e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010396975267212838 - rollout_corr/rollout_is_max:2.1670234203338623 - rollout_corr/rollout_is_min:0.16362474858760834 - rollout_corr/rollout_is_std:0.030536126345396042 - rollout_corr/rollout_is_eff_sample_size:0.999068294612318 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.0025593643076717854 - rollout_corr/rollout_is_seq_max:1.008395791053772 - rollout_corr/rollout_is_seq_min:0.9886937141418457 - rollout_corr/rollout_is_seq_max_deviation:0.011306285858154297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1031886860728264) - rollout_corr/training_log_ppl:np.float64(0.09535550630971557) - rollout_corr/kl:np.float64(0.000747708574247774) - rollout_corr/k3_kl:np.float64(0.0006307248504953122) - rollout_corr/rollout_ppl:np.float64(1.1023486168123782) - rollout_corr/rollout_log_ppl:np.float64(0.09460779764049221) - rollout_corr/log_ppl_diff:np.float64(0.000747708669223357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018803431230480783) - rollout_corr/log_ppl_diff_max:np.float64(0.013932567089796066) - rollout_corr/log_ppl_diff_min:np.float64(-0.007006004452705383) - rollout_corr/ppl_ratio:np.float64(1.0007511703297496) - rollout_corr/chi2_token:np.float64(0.000999337062239647) - rollout_corr/chi2_seq:np.float64(0.17764197965152562) - actor/pg_loss:np.float64(0.001810003537684679) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005623773683964828) - actor/ppo_kl:np.float64(0.000197975718934984) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.72265625) - self_distillation/token_reweight_w_mean:np.float64(103010.1768341891) - self_distillation/token_reweight_w_std:np.float64(1687668.886640767) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000071118120104) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03546456596814096) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.2238437756896019) - perf/mfu/actor:np.float64(0.03919303431293418) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.48588943481445) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027670131530612707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027670131530612707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:413.28125 - response_length/max:1808.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.28125 - response_length_non_aborted/max:1808.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.203617572784424e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7867520544677973) - timing_s/agent_loop/generate_sequences/max:np.float64(11.670810116454959) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.985955903917784) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.670810116454959) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1808 - timing_s/gen:17.77034630998969 - timing_s/reward:0.08232808858156204 - timing_s/old_log_prob:2.3802652321755886 - timing_s/adv:0.6049736253917217 - timing_s/update_actor:15.622556304559112 - timing_s/step:36.56155405379832 - timing_s/stop_profile:5.652382969856262e-05 - timing_per_token_ms/update_actor:0.09037902244966395 - timing_per_token_ms/adv:0.003499870559261592 - timing_per_token_ms/gen:0.167961685349619 - perf/total_num_tokens:172856 - perf/time_per_step:36.56155405379832 - perf/throughput:590.9759735104938 (TaskRunner pid=2074004) Training Progress: 85%|████████▌ | 85/100 [1:11:14<10:22, 41.47s/it] (TaskRunner pid=2074004) step:86 - global_seqlen/min:17265 - global_seqlen/max:27278 - global_seqlen/minmax_diff:10013 - global_seqlen/balanced_min:22005 - global_seqlen/balanced_max:22347 - global_seqlen/mean:22060.875 - actor/entropy:0.1164645105600357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5546000003814697 - training/rollout_probs_diff_mean:0.0027262147050350904 - training/rollout_probs_diff_std:0.011048424988985062 - training/rollout_actor_probs_pearson_corr:0.9979538917541504 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999389052391052 - rollout_corr/rollout_is_ratio_fraction_high:2.80670246866066e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.6778372052358463e-05 - rollout_corr/rollout_is_max:2.691387176513672 - rollout_corr/rollout_is_min:0.28650444746017456 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990854290712073 - rollout_corr/rollout_is_seq_mean:1.0000141859054565 - rollout_corr/rollout_is_seq_std:0.001993668731302023 - rollout_corr/rollout_is_seq_max:1.0104111433029175 - rollout_corr/rollout_is_seq_min:0.9925797581672668 - rollout_corr/rollout_is_seq_max_deviation:0.01041114330291748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.118015536107123) - rollout_corr/training_log_ppl:np.float64(0.10848677079411573) - rollout_corr/kl:np.float64(0.0006072961230856322) - rollout_corr/k3_kl:np.float64(0.0005712031893807534) - rollout_corr/rollout_ppl:np.float64(1.1173007288016379) - rollout_corr/rollout_log_ppl:np.float64(0.10787947439894197) - rollout_corr/log_ppl_diff:np.float64(0.000607296395173762) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015548389492323622) - rollout_corr/log_ppl_diff_max:np.float64(0.008465766906738281) - rollout_corr/log_ppl_diff_min:np.float64(-0.008714862167835236) - rollout_corr/ppl_ratio:np.float64(1.0006095981225371) - rollout_corr/chi2_token:np.float64(0.0010650106705725193) - rollout_corr/chi2_seq:np.float64(0.39449123083613813) - actor/pg_loss:np.float64(0.000594716053456068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002205379737461044) - actor/ppo_kl:np.float64(0.00013570546891372715) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(10468.989274371881) - self_distillation/token_reweight_w_std:np.float64(246581.90501564916) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998316585551947) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015069760556798428) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.09993577841669321) - perf/mfu/actor:np.float64(0.04011440240529307) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.5170135498047) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0022991569712758064 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0022991569712758064 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.52734375 - response_length/max:1781.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.52734375 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.181055843830109e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.044587777927518) - timing_s/agent_loop/generate_sequences/max:np.float64(11.397420486435294) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3413470640807645) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.397420486435294) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:17.311374351382256 - timing_s/reward:0.062451690435409546 - timing_s/old_log_prob:2.356656324118376 - timing_s/adv:0.579136572778225 - timing_s/update_actor:15.45087962038815 - timing_s/step:35.84615935757756 - timing_s/stop_profile:0.00011509284377098083 - timing_per_token_ms/update_actor:0.08754684265916556 - timing_per_token_ms/adv:0.0032814687358175103 - timing_per_token_ms/gen:0.16195958677278113 - perf/total_num_tokens:176487 - perf/time_per_step:35.84615935757756 - perf/throughput:615.4320405691252 (TaskRunner pid=2074004) Training Progress: 86%|████████▌ | 86/100 [1:11:50<09:17, 39.80s/it] (TaskRunner pid=2074004) step:87 - global_seqlen/min:18840 - global_seqlen/max:25215 - global_seqlen/minmax_diff:6375 - global_seqlen/balanced_min:22425 - global_seqlen/balanced_max:22454 - global_seqlen/mean:22447.0 - actor/entropy:0.12408103793859482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33947664499282837 - training/rollout_probs_diff_mean:0.0029533642809838057 - training/rollout_probs_diff_std:0.011205787770450115 - training/rollout_actor_probs_pearson_corr:0.9980510473251343 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000004768371582 - rollout_corr/rollout_is_ratio_fraction_high:9.21557057154132e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.372456457233056e-05 - rollout_corr/rollout_is_max:2.10186767578125 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03159523010253906 - rollout_corr/rollout_is_eff_sample_size:0.9990027369357228 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002129511907696724 - rollout_corr/rollout_is_seq_max:1.0119233131408691 - rollout_corr/rollout_is_seq_min:0.9937775135040283 - rollout_corr/rollout_is_seq_max_deviation:0.01192331314086914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1291866833344102) - rollout_corr/training_log_ppl:np.float64(0.11755550177258556) - rollout_corr/kl:np.float64(0.0004603647517873455) - rollout_corr/k3_kl:np.float64(0.0006310233444892788) - rollout_corr/rollout_ppl:np.float64(1.1286979308351874) - rollout_corr/rollout_log_ppl:np.float64(0.11709513665482518) - rollout_corr/log_ppl_diff:np.float64(0.00046036511776037514) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016326033364748582) - rollout_corr/log_ppl_diff_max:np.float64(0.007639244198799133) - rollout_corr/log_ppl_diff_min:np.float64(-0.010535374283790588) - rollout_corr/ppl_ratio:np.float64(1.0004629627801478) - rollout_corr/chi2_token:np.float64(0.0016186770517379045) - rollout_corr/chi2_seq:np.float64(0.7282668207772076) - actor/pg_loss:np.float64(0.0011052467161789536) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023867743630034965) - actor/ppo_kl:np.float64(1.1898970175394652e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(55895.11731084273) - self_distillation/token_reweight_w_std:np.float64(747223.8999097699) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997360575944185) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02877963568607811) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.2662094905972481) - perf/mfu/actor:np.float64(0.04090594702599697) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4480094909668) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027278088964521885 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027278088964521885 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:423.875 - response_length/max:1575.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.875 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011235661804676056 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7568011898547411) - timing_s/agent_loop/generate_sequences/max:np.float64(10.530228452757001) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.055000598782499) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.530228452757001) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:16.49220446869731 - timing_s/reward:0.06013626419007778 - timing_s/old_log_prob:2.299874674528837 - timing_s/adv:0.572662265971303 - timing_s/update_actor:15.511149242520332 - timing_s/step:34.9925875030458 - timing_s/stop_profile:0.00011404044926166534 - timing_per_token_ms/update_actor:0.0863765160295381 - timing_per_token_ms/adv:0.0031889688264094477 - timing_per_token_ms/gen:0.1519850750948956 - perf/total_num_tokens:179576 - perf/time_per_step:34.9925875030458 - perf/throughput:641.4787131144899 (TaskRunner pid=2074004) Training Progress: 87%|████████▋ | 87/100 [1:12:25<08:18, 38.37s/it] (TaskRunner pid=2074004) step:88 - global_seqlen/min:17924 - global_seqlen/max:32180 - global_seqlen/minmax_diff:14256 - global_seqlen/balanced_min:22814 - global_seqlen/balanced_max:22832 - global_seqlen/mean:22825.125 - actor/entropy:0.10808072239160538 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2607862949371338 - training/rollout_probs_diff_mean:0.0027171641122549772 - training/rollout_probs_diff_std:0.010765996761620045 - training/rollout_actor_probs_pearson_corr:0.9980359077453613 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999313950538635 - rollout_corr/rollout_is_ratio_fraction_high:1.7341692000627518e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3354230001568794e-05 - rollout_corr/rollout_is_max:3.181804656982422 - rollout_corr/rollout_is_min:0.01616358757019043 - rollout_corr/rollout_is_std:0.030833328142762184 - rollout_corr/rollout_is_eff_sample_size:0.9990502088766346 - rollout_corr/rollout_is_seq_mean:0.9999111890792847 - rollout_corr/rollout_is_seq_std:0.001777790836058557 - rollout_corr/rollout_is_seq_max:1.0082212686538696 - rollout_corr/rollout_is_seq_min:0.9945080876350403 - rollout_corr/rollout_is_seq_max_deviation:0.008221268653869629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.112278978805989) - rollout_corr/training_log_ppl:np.float64(0.10384648678154917) - rollout_corr/kl:np.float64(0.0007002655332808416) - rollout_corr/k3_kl:np.float64(0.0007456282953626214) - rollout_corr/rollout_ppl:np.float64(1.111463341396302) - rollout_corr/rollout_log_ppl:np.float64(0.10314622065925505) - rollout_corr/log_ppl_diff:np.float64(0.0007002661222941242) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017377006079186685) - rollout_corr/log_ppl_diff_max:np.float64(0.015303656458854675) - rollout_corr/log_ppl_diff_min:np.float64(-0.006968945264816284) - rollout_corr/ppl_ratio:np.float64(1.0007034207228571) - rollout_corr/chi2_token:np.float64(0.0014822578523308039) - rollout_corr/chi2_seq:np.float64(1.0283904024399817) - actor/pg_loss:np.float64(0.0028022772166877985) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011713389812939567) - actor/ppo_kl:np.float64(6.31784731428553e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(102159.92961579212) - self_distillation/token_reweight_w_std:np.float64(1634106.4369794251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001399826724082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04912651109043509) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.34347517415881157) - perf/mfu/actor:np.float64(0.041204584153654134) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4816131591797) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009243122301995754 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009243122301995754 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:450.50390625 - response_length/max:1537.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:450.50390625 - response_length_non_aborted/max:1537.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010040774941444397 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8878467436879873) - timing_s/agent_loop/generate_sequences/max:np.float64(10.596595136448741) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.400526633304253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.596595136448741) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1537 - timing_s/gen:16.373642224818468 - timing_s/reward:0.06224523112177849 - timing_s/old_log_prob:2.349311839789152 - timing_s/adv:0.6097414214164019 - timing_s/update_actor:15.592103796079755 - timing_s/step:35.07431404106319 - timing_s/stop_profile:0.00012129172682762146 - timing_per_token_ms/update_actor:0.08538892884529524 - timing_per_token_ms/adv:0.003339200888365353 - timing_per_token_ms/gen:0.1419733304270259 - perf/total_num_tokens:182601 - perf/time_per_step:35.07431404106319 - perf/throughput:650.764687037857 (TaskRunner pid=2074004) Training Progress: 88%|████████▊ | 88/100 [1:13:00<07:28, 37.40s/it] (TaskRunner pid=2074004) step:89 - global_seqlen/min:13752 - global_seqlen/max:27817 - global_seqlen/minmax_diff:14065 - global_seqlen/balanced_min:22301 - global_seqlen/balanced_max:22323 - global_seqlen/mean:22317.25 - actor/entropy:0.10688827931880951 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.601607620716095 - training/rollout_probs_diff_mean:0.002714034402742982 - training/rollout_probs_diff_std:0.011373176239430904 - training/rollout_actor_probs_pearson_corr:0.9978145360946655 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:1.0 - self_distillation/correct_sample_fraction:0.8125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999041557312012 - rollout_corr/rollout_is_ratio_fraction_high:3.661863593151793e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.239193266490474e-05 - rollout_corr/rollout_is_max:2.066577672958374 - rollout_corr/rollout_is_min:0.30741891264915466 - rollout_corr/rollout_is_std:0.030931759625673294 - rollout_corr/rollout_is_eff_sample_size:0.9990437838386964 - rollout_corr/rollout_is_seq_mean:0.9999012351036072 - rollout_corr/rollout_is_seq_std:0.00172822130843997 - rollout_corr/rollout_is_seq_max:1.0071568489074707 - rollout_corr/rollout_is_seq_min:0.9942929744720459 - rollout_corr/rollout_is_seq_max_deviation:0.007156848907470703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1084096948616207) - rollout_corr/training_log_ppl:np.float64(0.09965023837503395) - rollout_corr/kl:np.float64(0.0006493180134174992) - rollout_corr/k3_kl:np.float64(0.0006724119927685734) - rollout_corr/rollout_ppl:np.float64(1.107654474209994) - rollout_corr/rollout_log_ppl:np.float64(0.09900091987219639) - rollout_corr/log_ppl_diff:np.float64(0.0006493185028375592) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017076216572604608) - rollout_corr/log_ppl_diff_max:np.float64(0.01190245896577835) - rollout_corr/log_ppl_diff_min:np.float64(-0.013669520616531372) - rollout_corr/ppl_ratio:np.float64(1.0006527956575155) - rollout_corr/chi2_token:np.float64(0.0013369775842875242) - rollout_corr/chi2_seq:np.float64(5.031164270127192) - actor/pg_loss:np.float64(0.002956106443889439) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008950727274168457) - actor/ppo_kl:np.float64(4.916048081948787e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_w_mean:np.float64(71424.80499303155) - self_distillation/token_reweight_w_std:np.float64(1224926.4624935077) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000664854887873) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04491751454042969) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.3388006314635277) - perf/mfu/actor:np.float64(0.04076402686351015) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.86504364013672) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.8125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008070976473391056 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.008070976473391056 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:426.6953125 - response_length/max:1575.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.6953125 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003277640789747238 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9218454416841269) - timing_s/agent_loop/generate_sequences/max:np.float64(10.495202466845512) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.246527711606177) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.495202466845512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:16.446799524128437 - timing_s/reward:0.056499673053622246 - timing_s/old_log_prob:2.5402090717107058 - timing_s/adv:0.6734065376222134 - timing_s/update_actor:15.645092817023396 - timing_s/step:35.48852607421577 - timing_s/stop_profile:0.00012188777327537537 - timing_per_token_ms/update_actor:0.08762892390988695 - timing_per_token_ms/adv:0.003771782688403664 - timing_per_token_ms/gen:0.15056483809188015 - perf/total_num_tokens:178538 - perf/time_per_step:35.48852607421577 - perf/throughput:628.8581823130328 (TaskRunner pid=2074004) Training Progress: 89%|████████▉ | 89/100 [1:13:36<06:46, 36.99s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We are to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is **B: 56 pC**. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2074004) step:90 - global_seqlen/min:17503 - global_seqlen/max:30542 - global_seqlen/minmax_diff:13039 - global_seqlen/balanced_min:22163 - global_seqlen/balanced_max:22749 - global_seqlen/mean:22249.5 - actor/entropy:0.10518425703048706 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40461307764053345 - training/rollout_probs_diff_mean:0.0026247852947562933 - training/rollout_probs_diff_std:0.010737664066255093 - training/rollout_actor_probs_pearson_corr:0.9979662895202637 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0000884532928467 - rollout_corr/rollout_is_ratio_fraction_high:1.823819002311211e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.20718560134992e-05 - rollout_corr/rollout_is_max:2.0903313159942627 - rollout_corr/rollout_is_min:0.19070646166801453 - rollout_corr/rollout_is_std:0.029787156730890274 - rollout_corr/rollout_is_eff_sample_size:0.9991136308144901 - rollout_corr/rollout_is_seq_mean:1.0000239610671997 - rollout_corr/rollout_is_seq_std:0.00162831402849406 - rollout_corr/rollout_is_seq_max:1.0044366121292114 - rollout_corr/rollout_is_seq_min:0.9923098683357239 - rollout_corr/rollout_is_seq_max_deviation:0.007690131664276123 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.102782507892698) - rollout_corr/training_log_ppl:np.float64(0.09468803418349125) - rollout_corr/kl:np.float64(0.0005543236077372171) - rollout_corr/k3_kl:np.float64(0.0005838268394882107) - rollout_corr/rollout_ppl:np.float64(1.1021574116311967) - rollout_corr/rollout_log_ppl:np.float64(0.0941337100121018) - rollout_corr/log_ppl_diff:np.float64(0.0005543241713894531) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015017515033832751) - rollout_corr/log_ppl_diff_max:np.float64(0.010810032486915588) - rollout_corr/log_ppl_diff_min:np.float64(-0.006302401423454285) - rollout_corr/ppl_ratio:np.float64(1.0005565818864852) - rollout_corr/chi2_token:np.float64(0.0012237117625772953) - rollout_corr/chi2_seq:np.float64(0.7812420942354947) - actor/pg_loss:np.float64(0.001676898100413382) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036616270745071233) - actor/ppo_kl:np.float64(0.00012324695833854094) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_w_mean:np.float64(32326.37215174688) - self_distillation/token_reweight_w_std:np.float64(704860.9180972856) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991597740445286) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029383839399088174) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.14125302247703075) - perf/mfu/actor:np.float64(0.0403704412148885) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.92715072631836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/reward/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08294761183043528) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.821725) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0733318252008867) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.729125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.060664540093849084) - val-aux/sciknoweval/score/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/score/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08294761183043528) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.821725) - val-aux/sciknoweval/score/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0733318252008867) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.729125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.060664540093849084) - val-core/sciknoweval/acc/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/acc/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08294761183043528) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.821725) - val-core/sciknoweval/acc/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0733318252008867) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.729125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.060664540093849084) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009077304717673634) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.002405465425136726) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9957625) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.011852098487061466) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9977625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.008832748449106717) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9916125000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.015594544271371125) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9995750000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.003952916592698636) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9849375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.018366765656865063) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0013474443894951921) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9760875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0180217269071149) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018307723104953766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018307723104953766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.359375 - response_length/max:2385.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.359375 - response_length_non_aborted/max:2385.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017513521015644073 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2588027901947498) - timing_s/agent_loop/generate_sequences/max:np.float64(14.54117732308805) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3901306394909625) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.54117732308805) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:2385 - timing_s/gen:21.131900370121002 - timing_s/reward:0.05953768454492092 - timing_s/old_log_prob:2.3052819333970547 - timing_s/adv:0.5906773693859577 - timing_s/update_actor:15.686287306249142 - timing_s/step:39.856287997215986 - timing_s/testing:87.98717088066041 - timing_s/save_checkpoint:6.872611632570624 - timing_s/stop_profile:4.615262150764465e-05 - timing_per_token_ms/update_actor:0.08812718997196084 - timing_per_token_ms/adv:0.0033184867602977465 - timing_per_token_ms/gen:0.19270381515703996 - perf/total_num_tokens:177996 - perf/time_per_step:39.856287997215986 - perf/throughput:558.24315604991 (TaskRunner pid=2074004) Training Progress: 90%|█████████ | 90/100 [1:15:51<11:03, 66.32s/it] (TaskRunner pid=2074004) step:91 - global_seqlen/min:17728 - global_seqlen/max:25869 - global_seqlen/minmax_diff:8141 - global_seqlen/balanced_min:21525 - global_seqlen/balanced_max:21553 - global_seqlen/mean:21545.25 - actor/entropy:0.08843715488910675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41412943601608276 - training/rollout_probs_diff_mean:0.002419212367385626 - training/rollout_probs_diff_std:0.010600846260786057 - training/rollout_actor_probs_pearson_corr:0.9976838231086731 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:9.281086931878235e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.712434772751294e-05 - rollout_corr/rollout_is_max:2.0842576026916504 - rollout_corr/rollout_is_min:0.18285298347473145 - rollout_corr/rollout_is_std:0.027998575940728188 - rollout_corr/rollout_is_eff_sample_size:0.999216455716407 - rollout_corr/rollout_is_seq_mean:0.9998674392700195 - rollout_corr/rollout_is_seq_std:0.0018235978204756975 - rollout_corr/rollout_is_seq_max:1.007836103439331 - rollout_corr/rollout_is_seq_min:0.9920446276664734 - rollout_corr/rollout_is_seq_max_deviation:0.007955372333526611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0833140267059207) - rollout_corr/training_log_ppl:np.float64(0.07899068885672023) - rollout_corr/kl:np.float64(0.0006408023299613319) - rollout_corr/k3_kl:np.float64(0.00047720418808694376) - rollout_corr/rollout_ppl:np.float64(1.0826310594566166) - rollout_corr/rollout_log_ppl:np.float64(0.07834988591639558) - rollout_corr/log_ppl_diff:np.float64(0.0006408029403246474) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014788467706239317) - rollout_corr/log_ppl_diff_max:np.float64(0.01340678334236145) - rollout_corr/log_ppl_diff_min:np.float64(-0.006584286689758301) - rollout_corr/ppl_ratio:np.float64(1.0006434579845518) - rollout_corr/chi2_token:np.float64(0.0005903118290007114) - rollout_corr/chi2_seq:np.float64(0.44575339392758906) - actor/pg_loss:np.float64(0.00038603588473051786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025614264700379863) - actor/ppo_kl:np.float64(7.664802197382414e-05) - actor/pg_clipfrac_lower:np.float64(5.7444854064669926e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(74129.82275516656) - self_distillation/token_reweight_w_std:np.float64(1313364.9157028943) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003251135349274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0171360018721316) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.11729132384061813) - perf/mfu/actor:np.float64(0.03910900715029075) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.49448776245117) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034270414616912603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034270414616912603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.8828125 - response_length/max:1701.0 - response_length/min:54.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.8828125 - response_length_non_aborted/max:1701.0 - response_length_non_aborted/min:54.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.895302355289459e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.649962367489934) - timing_s/agent_loop/generate_sequences/max:np.float64(11.452977204695344) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1154736819225946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.452977204695344) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1701 - timing_s/gen:17.09042201936245 - timing_s/reward:0.056309279054403305 - timing_s/old_log_prob:2.4856445379555225 - timing_s/adv:0.686548063531518 - timing_s/update_actor:15.663952223956585 - timing_s/step:36.09207380563021 - timing_s/stop_profile:0.00012318789958953857 - timing_per_token_ms/update_actor:0.0908782227170524 - timing_per_token_ms/adv:0.003983175314347234 - timing_per_token_ms/gen:0.1586176936439631 - perf/total_num_tokens:172362 - perf/time_per_step:36.09207380563021 - perf/throughput:596.95239780428 (TaskRunner pid=2074004) Training Progress: 91%|█████████ | 91/100 [1:16:27<08:35, 57.27s/it] (TaskRunner pid=2074004) step:92 - global_seqlen/min:18485 - global_seqlen/max:32646 - global_seqlen/minmax_diff:14161 - global_seqlen/balanced_min:24274 - global_seqlen/balanced_max:24304 - global_seqlen/mean:24296.5 - actor/entropy:0.1161329373717308 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46171876788139343 - training/rollout_probs_diff_mean:0.0028441078029572964 - training/rollout_probs_diff_std:0.011383011005818844 - training/rollout_actor_probs_pearson_corr:0.997846245765686 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.000044584274292 - rollout_corr/rollout_is_ratio_fraction_high:3.2485462725162506e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.060682840645313e-05 - rollout_corr/rollout_is_max:2.6116247177124023 - rollout_corr/rollout_is_min:0.3187040090560913 - rollout_corr/rollout_is_std:0.03276945650577545 - rollout_corr/rollout_is_eff_sample_size:0.998927552516703 - rollout_corr/rollout_is_seq_mean:0.9999802708625793 - rollout_corr/rollout_is_seq_std:0.0019448020029813051 - rollout_corr/rollout_is_seq_max:1.0066347122192383 - rollout_corr/rollout_is_seq_min:0.9897977113723755 - rollout_corr/rollout_is_seq_max_deviation:0.010202288627624512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.125696862116456) - rollout_corr/training_log_ppl:np.float64(0.11609659137320705) - rollout_corr/kl:np.float64(0.0009234349944136078) - rollout_corr/k3_kl:np.float64(0.0007315961058225184) - rollout_corr/rollout_ppl:np.float64(1.1246338286437094) - rollout_corr/rollout_log_ppl:np.float64(0.11517315577657428) - rollout_corr/log_ppl_diff:np.float64(0.0009234355966327712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018325499549973756) - rollout_corr/log_ppl_diff_max:np.float64(0.012700438499450684) - rollout_corr/log_ppl_diff_min:np.float64(-0.006820157170295715) - rollout_corr/ppl_ratio:np.float64(1.0009266710840166) - rollout_corr/chi2_token:np.float64(0.0010245046578347683) - rollout_corr/chi2_seq:np.float64(0.6241324651055038) - actor/pg_loss:np.float64(0.0013751451624557376) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006302250340013416) - actor/ppo_kl:np.float64(0.0003064914879740144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(70968.99266718118) - self_distillation/token_reweight_w_std:np.float64(1136299.2140333871) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998132719192654) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03681001826771535) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.22948458045721054) - perf/mfu/actor:np.float64(0.04381601917617662) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.27560424804688) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007187408627942204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007187408627942204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.984375 - response_length/max:1689.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.984375 - response_length_non_aborted/max:1689.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012553296983242035 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0251992959529161) - timing_s/agent_loop/generate_sequences/max:np.float64(11.716115767136216) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.778788086819986) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.716115767136216) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1689 - timing_s/gen:17.857277484610677 - timing_s/reward:0.061120714992284775 - timing_s/old_log_prob:2.40597465634346 - timing_s/adv:0.6833560299128294 - timing_s/update_actor:15.71237531118095 - timing_s/step:36.81252425536513 - timing_s/stop_profile:0.00011310726404190063 - timing_per_token_ms/update_actor:0.08083661901498647 - timing_per_token_ms/adv:0.003515712293503331 - timing_per_token_ms/gen:0.14502548065986645 - perf/total_num_tokens:194372 - perf/time_per_step:36.81252425536513 - perf/throughput:660.0063563003011 (TaskRunner pid=2074004) Training Progress: 92%|█████████▏| 92/100 [1:17:04<06:49, 51.14s/it] (TaskRunner pid=2074004) step:93 - global_seqlen/min:14252 - global_seqlen/max:27080 - global_seqlen/minmax_diff:12828 - global_seqlen/balanced_min:19571 - global_seqlen/balanced_max:25808 - global_seqlen/mean:20389.375 - actor/entropy:0.11118520796298981 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2569490671157837 - training/rollout_probs_diff_mean:0.0027298631612211466 - training/rollout_probs_diff_std:0.01081868913024664 - training/rollout_actor_probs_pearson_corr:0.9979827404022217 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999743103981018 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9497259855270386 - rollout_corr/rollout_is_min:0.5504913330078125 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990726971602333 - rollout_corr/rollout_is_seq_mean:1.000046968460083 - rollout_corr/rollout_is_seq_std:0.0025716039817780256 - rollout_corr/rollout_is_seq_max:1.0111936330795288 - rollout_corr/rollout_is_seq_min:0.9910350441932678 - rollout_corr/rollout_is_seq_max_deviation:0.011193633079528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1135128010064363) - rollout_corr/training_log_ppl:np.float64(0.1049198287037143) - rollout_corr/kl:np.float64(0.0006423072564984977) - rollout_corr/k3_kl:np.float64(0.0005943681892190966) - rollout_corr/rollout_ppl:np.float64(1.1127843372523785) - rollout_corr/rollout_log_ppl:np.float64(0.10427752112082089) - rollout_corr/log_ppl_diff:np.float64(0.0006423075828934088) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018248588894493878) - rollout_corr/log_ppl_diff_max:np.float64(0.009763795882463455) - rollout_corr/log_ppl_diff_min:np.float64(-0.008692525327205658) - rollout_corr/ppl_ratio:np.float64(1.0006455411203206) - rollout_corr/chi2_token:np.float64(0.0011548018082976341) - rollout_corr/chi2_seq:np.float64(0.28039377136155963) - actor/pg_loss:np.float64(0.0013577784411609173) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004785834780705045) - actor/ppo_kl:np.float64(0.0001363070303765923) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(101776.64841872128) - self_distillation/token_reweight_w_std:np.float64(1415721.968034971) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006784941069782) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037072227569296956) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1903113927692175) - perf/mfu/actor:np.float64(0.0352703899917267) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2206916809082) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06920255720615387 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06920255720615387 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:379.94921875 - response_length/max:8192.0 - response_length/min:72.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:379.94921875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.636208415031433e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.785060852766037) - timing_s/agent_loop/generate_sequences/max:np.float64(50.456341341137886) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6300632040947676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.456341341137886) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:280 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.36329558491707 - timing_s/reward:0.06057967245578766 - timing_s/old_log_prob:2.5846155025064945 - timing_s/adv:0.5508739780634642 - timing_s/update_actor:16.457574239000678 - timing_s/step:76.1049917601049 - timing_s/stop_profile:0.00011239387094974518 - timing_per_token_ms/update_actor:0.10089552916041246 - timing_per_token_ms/adv:0.0033772122616771245 - timing_per_token_ms/gen:0.5794698673231113 - perf/total_num_tokens:163115 - perf/time_per_step:76.1049917601049 - perf/throughput:267.91113865790265 (TaskRunner pid=2074004) Training Progress: 93%|█████████▎| 93/100 [1:18:20<06:50, 58.65s/it] (TaskRunner pid=2074004) step:94 - global_seqlen/min:16134 - global_seqlen/max:25356 - global_seqlen/minmax_diff:9222 - global_seqlen/balanced_min:19639 - global_seqlen/balanced_max:19643 - global_seqlen/mean:19641.0 - actor/entropy:0.09782024472951889 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7454107999801636 - training/rollout_probs_diff_mean:0.002670757472515106 - training/rollout_probs_diff_std:0.01144047174602747 - training/rollout_actor_probs_pearson_corr:0.9975129961967468 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.999981164932251 - rollout_corr/rollout_is_ratio_fraction_high:2.235536157968454e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010059912165161222 - rollout_corr/rollout_is_max:2.278707504272461 - rollout_corr/rollout_is_min:0.010440262034535408 - rollout_corr/rollout_is_std:0.03061799891293049 - rollout_corr/rollout_is_eff_sample_size:0.9990632971725152 - rollout_corr/rollout_is_seq_mean:0.9999101161956787 - rollout_corr/rollout_is_seq_std:0.0022188930306583643 - rollout_corr/rollout_is_seq_max:1.0089306831359863 - rollout_corr/rollout_is_seq_min:0.9909623265266418 - rollout_corr/rollout_is_seq_max_deviation:0.009037673473358154 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1009153043851256) - rollout_corr/training_log_ppl:np.float64(0.09386854151671287) - rollout_corr/kl:np.float64(0.0008569538749156891) - rollout_corr/k3_kl:np.float64(0.0006892196433412323) - rollout_corr/rollout_ppl:np.float64(1.0999419218860567) - rollout_corr/rollout_log_ppl:np.float64(0.093011587527144) - rollout_corr/log_ppl_diff:np.float64(0.0008569539895688649) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018225670246465597) - rollout_corr/log_ppl_diff_max:np.float64(0.012111961841583252) - rollout_corr/log_ppl_diff_min:np.float64(-0.009987182915210724) - rollout_corr/ppl_ratio:np.float64(1.0008604750037193) - rollout_corr/chi2_token:np.float64(0.0010376402642577887) - rollout_corr/chi2_seq:np.float64(0.17592160333879292) - actor/pg_loss:np.float64(0.0020128018222749233) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006171084928610071) - actor/ppo_kl:np.float64(0.00018625303451358377) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(47890.5027854254) - self_distillation/token_reweight_w_std:np.float64(780379.3705423808) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003406463656574) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03524100726644974) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.30077048763632774) - perf/mfu/actor:np.float64(0.03571617935583021) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.26049041748047) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005389038939028978 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005389038939028978 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.46875 - response_length/max:992.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.46875 - response_length_non_aborted/max:992.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011242926120758057 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7382768783718348) - timing_s/agent_loop/generate_sequences/max:np.float64(7.45197612978518) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5364193320201593) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.45197612978518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:992 - timing_s/gen:13.292611926794052 - timing_s/reward:0.058578019961714745 - timing_s/old_log_prob:2.304050751030445 - timing_s/adv:0.5737872198224068 - timing_s/update_actor:15.573226625099778 - timing_s/step:31.890709141269326 - timing_s/stop_profile:0.00011250190436840057 - timing_per_token_ms/update_actor:0.09911172181342459 - timing_per_token_ms/adv:0.0036517184704343386 - timing_per_token_ms/gen:0.14858056790210647 - perf/total_num_tokens:157128 - perf/time_per_step:31.890709141269326 - perf/throughput:615.8847052599044 (TaskRunner pid=2074004) Training Progress: 94%|█████████▍| 94/100 [1:18:52<05:03, 50.64s/it] (TaskRunner pid=2074004) step:95 - global_seqlen/min:19898 - global_seqlen/max:27127 - global_seqlen/minmax_diff:7229 - global_seqlen/balanced_min:22383 - global_seqlen/balanced_max:22390 - global_seqlen/mean:22386.125 - actor/entropy:0.11839713156223297 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4174291491508484 - training/rollout_probs_diff_mean:0.002880886197090149 - training/rollout_probs_diff_std:0.01125163584947586 - training/rollout_actor_probs_pearson_corr:0.9979742765426636 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000062108039856 - rollout_corr/rollout_is_ratio_fraction_high:1.846091436163988e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7691370632965118e-05 - rollout_corr/rollout_is_max:2.8053035736083984 - rollout_corr/rollout_is_min:0.35777518153190613 - rollout_corr/rollout_is_std:0.031417399644851685 - rollout_corr/rollout_is_eff_sample_size:0.9990140393695902 - rollout_corr/rollout_is_seq_mean:1.0001864433288574 - rollout_corr/rollout_is_seq_std:0.001943354494869709 - rollout_corr/rollout_is_seq_max:1.009150743484497 - rollout_corr/rollout_is_seq_min:0.9956559538841248 - rollout_corr/rollout_is_seq_max_deviation:0.00915074348449707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1184041863307357) - rollout_corr/training_log_ppl:np.float64(0.10852470160170924) - rollout_corr/kl:np.float64(0.0004109893218089411) - rollout_corr/k3_kl:np.float64(0.0005756196864368235) - rollout_corr/rollout_ppl:np.float64(1.1179265826940536) - rollout_corr/rollout_log_ppl:np.float64(0.10811371153613436) - rollout_corr/log_ppl_diff:np.float64(0.0004109900655748788) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015215025887300726) - rollout_corr/log_ppl_diff_max:np.float64(0.010581329464912415) - rollout_corr/log_ppl_diff_min:np.float64(-0.007771208882331848) - rollout_corr/ppl_ratio:np.float64(1.0004132050089538) - rollout_corr/chi2_token:np.float64(0.0015168595127761364) - rollout_corr/chi2_seq:np.float64(0.37046705558896065) - actor/pg_loss:np.float64(0.0014443554682657123) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039592202506355534) - actor/ppo_kl:np.float64(9.238178380144291e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(35129.66564378468) - self_distillation/token_reweight_w_std:np.float64(673946.7570342987) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999094212660566) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03862924790882971) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.22043711598962545) - perf/mfu/actor:np.float64(0.04063184804557128) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2927703857422) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033506557811051607 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033506557811051607 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.19140625 - response_length/max:1177.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.19140625 - response_length_non_aborted/max:1177.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.422190487384796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937835987657309) - timing_s/agent_loop/generate_sequences/max:np.float64(8.682577695697546) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320787707940326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.682577695697546) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1171 - timing_s/gen:14.465107165277004 - timing_s/reward:0.061005301773548126 - timing_s/old_log_prob:2.500914089381695 - timing_s/adv:0.5702372565865517 - timing_s/update_actor:15.484657960012555 - timing_s/step:33.17570861801505 - timing_s/stop_profile:0.00012160651385784149 - timing_per_token_ms/update_actor:0.08646347882903224 - timing_per_token_ms/adv:0.0031840998419029177 - timing_per_token_ms/gen:0.13351954701788865 - perf/total_num_tokens:179089 - perf/time_per_step:33.17570861801505 - perf/throughput:674.7745845538293 (TaskRunner pid=2074004) Training Progress: 95%|█████████▌| 95/100 [1:19:25<03:47, 45.41s/it] (TaskRunner pid=2074004) step:96 - global_seqlen/min:14700 - global_seqlen/max:31388 - global_seqlen/minmax_diff:16688 - global_seqlen/balanced_min:22938 - global_seqlen/balanced_max:22942 - global_seqlen/mean:22940.5 - actor/entropy:0.10098060220479965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6580971479415894 - training/rollout_probs_diff_mean:0.002621095860376954 - training/rollout_probs_diff_std:0.011137053370475769 - training/rollout_actor_probs_pearson_corr:0.9977489113807678 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001559257507324 - rollout_corr/rollout_is_ratio_fraction_high:1.786160828487482e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.251563172554597e-05 - rollout_corr/rollout_is_max:2.2557666301727295 - rollout_corr/rollout_is_min:0.252839058637619 - rollout_corr/rollout_is_std:0.03050878830254078 - rollout_corr/rollout_is_eff_sample_size:0.9990705553753199 - rollout_corr/rollout_is_seq_mean:1.0001717805862427 - rollout_corr/rollout_is_seq_std:0.001738636172376573 - rollout_corr/rollout_is_seq_max:1.007121205329895 - rollout_corr/rollout_is_seq_min:0.9931297898292542 - rollout_corr/rollout_is_seq_max_deviation:0.0071212053298950195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.103144368622452) - rollout_corr/training_log_ppl:np.float64(0.09644664141524117) - rollout_corr/kl:np.float64(0.0005017887852607483) - rollout_corr/k3_kl:np.float64(0.0005338758359130225) - rollout_corr/rollout_ppl:np.float64(1.1025838060304523) - rollout_corr/rollout_log_ppl:np.float64(0.09594485214620363) - rollout_corr/log_ppl_diff:np.float64(0.0005017892690375447) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013042025966569781) - rollout_corr/log_ppl_diff_max:np.float64(0.006274871528148651) - rollout_corr/log_ppl_diff_min:np.float64(-0.005490683019161224) - rollout_corr/ppl_ratio:np.float64(1.0005033176857978) - rollout_corr/chi2_token:np.float64(0.0011128755286335945) - rollout_corr/chi2_seq:np.float64(0.5927000071387738) - actor/pg_loss:np.float64(0.0014072328340262175) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000444404973222845) - actor/ppo_kl:np.float64(0.000197858096928627) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(106493.64426750923) - self_distillation/token_reweight_w_std:np.float64(2085160.6336320788) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998901744838804) - self_distillation/token_reweight_w_clip_frac:np.float64(0.036775027387193404) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2108887266367674) - perf/mfu/actor:np.float64(0.0416768245283575) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.38690567016602) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002023943467065692 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002023943467065692 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.390625 - response_length/max:1046.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.390625 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.732507169246674e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1755850836634636) - timing_s/agent_loop/generate_sequences/max:np.float64(8.23095478862524) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5998671856941655) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.23095478862524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:14.00526099652052 - timing_s/reward:0.06283171847462654 - timing_s/old_log_prob:2.3491326589137316 - timing_s/adv:0.5709024202078581 - timing_s/update_actor:15.67631054110825 - timing_s/step:32.75171426124871 - timing_s/stop_profile:0.0001319441944360733 - timing_per_token_ms/update_actor:0.08541831336015045 - timing_per_token_ms/adv:0.0031107779920220686 - timing_per_token_ms/gen:0.1250782427439049 - perf/total_num_tokens:183524 - perf/time_per_step:32.75171426124871 - perf/throughput:700.4366188899866 (TaskRunner pid=2074004) Training Progress: 96%|█████████▌| 96/100 [1:19:58<02:46, 41.63s/it] (TaskRunner pid=2074004) step:97 - global_seqlen/min:17529 - global_seqlen/max:26956 - global_seqlen/minmax_diff:9427 - global_seqlen/balanced_min:22453 - global_seqlen/balanced_max:22536 - global_seqlen/mean:22478.25 - actor/entropy:0.11855632066726685 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018026649951935 - training/rollout_probs_diff_mean:0.0028152333106845617 - training/rollout_probs_diff_std:0.010734545066952705 - training/rollout_actor_probs_pearson_corr:0.9980775713920593 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.79296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:0.9999098777770996 - rollout_corr/rollout_is_ratio_fraction_high:1.8008608094532974e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4025826102588326e-05 - rollout_corr/rollout_is_max:4.380692005157471 - rollout_corr/rollout_is_min:0.2371114045381546 - rollout_corr/rollout_is_std:0.030879687517881393 - rollout_corr/rollout_is_eff_sample_size:0.9990471153295336 - rollout_corr/rollout_is_seq_mean:0.9999960660934448 - rollout_corr/rollout_is_seq_std:0.0019316772231832147 - rollout_corr/rollout_is_seq_max:1.0092357397079468 - rollout_corr/rollout_is_seq_min:0.9920583367347717 - rollout_corr/rollout_is_seq_max_deviation:0.009235739707946777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1194870863109827) - rollout_corr/training_log_ppl:np.float64(0.10984483735228423) - rollout_corr/kl:np.float64(0.0007211861242026885) - rollout_corr/k3_kl:np.float64(0.0006180598990965791) - rollout_corr/rollout_ppl:np.float64(1.118659039027989) - rollout_corr/rollout_log_ppl:np.float64(0.10912365088734077) - rollout_corr/log_ppl_diff:np.float64(0.0007211864649434574) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015570802715956233) - rollout_corr/log_ppl_diff_max:np.float64(0.011463932693004608) - rollout_corr/log_ppl_diff_min:np.float64(-0.005851447582244873) - rollout_corr/ppl_ratio:np.float64(1.000723579665646) - rollout_corr/chi2_token:np.float64(0.0009887183550745249) - rollout_corr/chi2_seq:np.float64(0.33689292683266103) - actor/pg_loss:np.float64(0.0028610387817025185) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005207407527905161) - actor/ppo_kl:np.float64(0.00018532980526941856) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_w_mean:np.float64(61391.32670080848) - self_distillation/token_reweight_w_std:np.float64(1146314.7785266174) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994598587509245) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060132228885777295) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.27978259697556496) - perf/mfu/actor:np.float64(0.04065052381475308) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.33649063110352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006340155377984047 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006340155377984047 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.8203125 - response_length/max:1625.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.8203125 - response_length_non_aborted/max:1625.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014923326671123505 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1448223646730185) - timing_s/agent_loop/generate_sequences/max:np.float64(10.64167152158916) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.490232729578565) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.64167152158916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1625 - timing_s/gen:16.820046979933977 - timing_s/reward:0.062204478308558464 - timing_s/old_log_prob:2.3776866644620895 - timing_s/adv:0.6371833365410566 - timing_s/update_actor:15.58851253055036 - timing_s/step:35.57156837172806 - timing_s/stop_profile:0.00011635571718215942 - timing_per_token_ms/update_actor:0.08668664448161201 - timing_per_token_ms/adv:0.003543332646786653 - timing_per_token_ms/gen:0.15145281726605897 - perf/total_num_tokens:179826 - perf/time_per_step:35.57156837172806 - perf/throughput:631.9161911867089 (TaskRunner pid=2074004) Training Progress: 97%|█████████▋| 97/100 [1:20:34<01:59, 39.83s/it] (TaskRunner pid=2074004) step:98 - global_seqlen/min:17481 - global_seqlen/max:26054 - global_seqlen/minmax_diff:8573 - global_seqlen/balanced_min:22060 - global_seqlen/balanced_max:22069 - global_seqlen/mean:22065.375 - actor/entropy:0.09252151846885681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6313449740409851 - training/rollout_probs_diff_mean:0.0024829101748764515 - training/rollout_probs_diff_std:0.01076439581811428 - training/rollout_actor_probs_pearson_corr:0.9977108836174011 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999709129333496 - rollout_corr/rollout_is_ratio_fraction_high:2.7911390134249814e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.582278026849963e-05 - rollout_corr/rollout_is_max:2.398716926574707 - rollout_corr/rollout_is_min:0.06967287510633469 - rollout_corr/rollout_is_std:0.02976113185286522 - rollout_corr/rollout_is_eff_sample_size:0.9991149397948247 - rollout_corr/rollout_is_seq_mean:0.9999794363975525 - rollout_corr/rollout_is_seq_std:0.0017344678053632379 - rollout_corr/rollout_is_seq_max:1.0062588453292847 - rollout_corr/rollout_is_seq_min:0.9945258498191833 - rollout_corr/rollout_is_seq_max_deviation:0.006258845329284668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.093379558995366) - rollout_corr/training_log_ppl:np.float64(0.08759709404694149) - rollout_corr/kl:np.float64(0.000628665536446249) - rollout_corr/k3_kl:np.float64(0.0005191412595024758) - rollout_corr/rollout_ppl:np.float64(1.0926776281557977) - rollout_corr/rollout_log_ppl:np.float64(0.08696842801145976) - rollout_corr/log_ppl_diff:np.float64(0.0006286660354817286) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013417766604106873) - rollout_corr/log_ppl_diff_max:np.float64(0.00907956063747406) - rollout_corr/log_ppl_diff_min:np.float64(-0.005724966526031494) - rollout_corr/ppl_ratio:np.float64(1.0006304366979748) - rollout_corr/chi2_token:np.float64(0.000801675021648407) - rollout_corr/chi2_seq:np.float64(0.10571352113038301) - actor/pg_loss:np.float64(0.0008303822251036763) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003051055082323728) - actor/ppo_kl:np.float64(0.00013278382333581362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(32778.418274484575) - self_distillation/token_reweight_w_std:np.float64(637282.1950311251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001593332272023) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02661549543699948) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.16087789833545685) - perf/mfu/actor:np.float64(0.03977215253160561) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.22827196121216) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006000949069857597 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006000949069857597 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:419.85546875 - response_length/max:1305.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:419.85546875 - response_length_non_aborted/max:1305.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011768564581871033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7680980022996664) - timing_s/agent_loop/generate_sequences/max:np.float64(9.386873960494995) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.251178185098979) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.386873960494995) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1305 - timing_s/gen:15.361656874418259 - timing_s/reward:0.06246807053685188 - timing_s/old_log_prob:2.382600510492921 - timing_s/adv:0.5709490403532982 - timing_s/update_actor:15.436897011473775 - timing_s/step:33.90267143212259 - timing_s/stop_profile:0.00010995566844940186 - timing_per_token_ms/update_actor:0.0874497771478718 - timing_per_token_ms/adv:0.003234417273405155 - timing_per_token_ms/gen:0.14292173529226257 - perf/total_num_tokens:176523 - perf/time_per_step:33.90267143212259 - perf/throughput:650.8447289818342 (TaskRunner pid=2074004) Training Progress: 98%|█████████▊| 98/100 [1:21:08<01:16, 38.07s/it] (TaskRunner pid=2074004) step:99 - global_seqlen/min:17883 - global_seqlen/max:26736 - global_seqlen/minmax_diff:8853 - global_seqlen/balanced_min:21621 - global_seqlen/balanced_max:21644 - global_seqlen/mean:21636.5 - actor/entropy:0.10580387711524963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4862961173057556 - training/rollout_probs_diff_mean:0.002484919736161828 - training/rollout_probs_diff_std:0.0104207843542099 - training/rollout_actor_probs_pearson_corr:0.9981136918067932 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96875 - self_distillation/correct_sample_fraction:0.90234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96875 - rollout_corr/rollout_is_mean:0.9999649524688721 - rollout_corr/rollout_is_ratio_fraction_high:9.457158739678562e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.674295607605018e-05 - rollout_corr/rollout_is_max:2.7389602661132812 - rollout_corr/rollout_is_min:0.13493947684764862 - rollout_corr/rollout_is_std:0.028364475816488266 - rollout_corr/rollout_is_eff_sample_size:0.9991959842570144 - rollout_corr/rollout_is_seq_mean:0.9998395442962646 - rollout_corr/rollout_is_seq_std:0.001808670349419117 - rollout_corr/rollout_is_seq_max:1.007258415222168 - rollout_corr/rollout_is_seq_min:0.9866185188293457 - rollout_corr/rollout_is_seq_max_deviation:0.013381481170654297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0939670382067561) - rollout_corr/training_log_ppl:np.float64(0.08732558643532684) - rollout_corr/kl:np.float64(0.000521703354597669) - rollout_corr/k3_kl:np.float64(0.00046708716944898754) - rollout_corr/rollout_ppl:np.float64(1.0933743044734001) - rollout_corr/rollout_log_ppl:np.float64(0.0868038823173265) - rollout_corr/log_ppl_diff:np.float64(0.0005217041180003434) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012858145564678125) - rollout_corr/log_ppl_diff_max:np.float64(0.015529707074165344) - rollout_corr/log_ppl_diff_min:np.float64(-0.006662759929895401) - rollout_corr/ppl_ratio:np.float64(1.000523641007021) - rollout_corr/chi2_token:np.float64(0.0009014918468892574) - rollout_corr/chi2_seq:np.float64(0.2561511362437159) - actor/pg_loss:np.float64(0.0010093091987073421) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018811171594279585) - actor/ppo_kl:np.float64(-4.058463273981516e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_w_mean:np.float64(28957.405509632314) - self_distillation/token_reweight_w_std:np.float64(466637.31905894005) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997467994689941) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0230792346119415) - self_distillation/empty_target_batch:np.float64(0.03125) - actor/grad_norm:np.float64(0.24430519342422485) - perf/mfu/actor:np.float64(0.039178741434394745) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.19931030273438) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.90234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.90234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005199073348194361 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005199073348194361 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:413.046875 - response_length/max:1520.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.046875 - response_length_non_aborted/max:1520.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013734027743339539 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8036548234522343) - timing_s/agent_loop/generate_sequences/max:np.float64(10.340177113190293) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.11781918026827) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.340177113190293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:1520 - timing_s/gen:16.068710835650563 - timing_s/reward:0.06060960330069065 - timing_s/old_log_prob:2.314846996217966 - timing_s/adv:0.6209873277693987 - timing_s/update_actor:15.458110202103853 - timing_s/step:34.610270539298654 - timing_s/stop_profile:0.0001095551997423172 - timing_per_token_ms/update_actor:0.08930574608938514 - timing_per_token_ms/adv:0.0035876142616030706 - timing_per_token_ms/gen:0.15196435441318862 - perf/total_num_tokens:173092 - perf/time_per_step:34.610270539298654 - perf/throughput:625.1468036180928 (TaskRunner pid=2074004) Training Progress: 99%|█████████▉| 99/100 [1:21:42<00:37, 37.05s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2074004) step:100 - global_seqlen/min:14131 - global_seqlen/max:24831 - global_seqlen/minmax_diff:10700 - global_seqlen/balanced_min:20764 - global_seqlen/balanced_max:20778 - global_seqlen/mean:20773.375 - actor/entropy:0.10674719512462616 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6354759931564331 - training/rollout_probs_diff_mean:0.0026373525615781546 - training/rollout_probs_diff_std:0.011061684228479862 - training/rollout_actor_probs_pearson_corr:0.9978691935539246 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0000841617584229 - rollout_corr/rollout_is_ratio_fraction_high:3.084801210206933e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.169602420413867e-05 - rollout_corr/rollout_is_max:3.1517021656036377 - rollout_corr/rollout_is_min:0.13044416904449463 - rollout_corr/rollout_is_std:0.03166119009256363 - rollout_corr/rollout_is_eff_sample_size:0.998998810886964 - rollout_corr/rollout_is_seq_mean:1.0001347064971924 - rollout_corr/rollout_is_seq_std:0.0022613920737057924 - rollout_corr/rollout_is_seq_max:1.0161210298538208 - rollout_corr/rollout_is_seq_min:0.9922299981117249 - rollout_corr/rollout_is_seq_max_deviation:0.0161210298538208 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1050673802383244) - rollout_corr/training_log_ppl:np.float64(0.09719672136634472) - rollout_corr/kl:np.float64(0.00043732599254298066) - rollout_corr/k3_kl:np.float64(0.0005812929390707211) - rollout_corr/rollout_ppl:np.float64(1.1045606597326696) - rollout_corr/rollout_log_ppl:np.float64(0.09675939454609761) - rollout_corr/log_ppl_diff:np.float64(0.0004373268202471081) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015722126299806405) - rollout_corr/log_ppl_diff_max:np.float64(0.009256497025489807) - rollout_corr/log_ppl_diff_min:np.float64(-0.006794825196266174) - rollout_corr/ppl_ratio:np.float64(1.0004396380390972) - rollout_corr/chi2_token:np.float64(0.0014464769046753645) - rollout_corr/chi2_seq:np.float64(0.31380061944946647) - actor/pg_loss:np.float64(0.0012798310490325093) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005447886960610049) - actor/ppo_kl:np.float64(1.8937110884076347e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(181507.5251128471) - self_distillation/token_reweight_w_std:np.float64(3188081.7809230387) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007254967931658) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029081717497319914) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.18284212984144688) - perf/mfu/actor:np.float64(0.038138719584986394) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.18253707885742) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/reward/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0934240016452198) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8469249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08389710095714561) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7487375000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07219685215265889) - val-aux/sciknoweval/score/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/score/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0934240016452198) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8469249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08389710095714561) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7487375000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07219685215265889) - val-core/sciknoweval/acc/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/acc/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0934240016452198) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8469249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08389710095714561) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7487375000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07219685215265889) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.006051536478449089) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.001440128696321335) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9969625000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.008153644123697198) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9985125) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.005913422059734724) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9942624999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.010511931298848583) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.002649136032174741) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9901125000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.012221532639586832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9841000000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.012028687586990721) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031310732010751963 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0031310732010751963 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:379.88671875 - response_length/max:1422.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:379.88671875 - response_length_non_aborted/max:1422.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011692196130752563 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5598204713314772) - timing_s/agent_loop/generate_sequences/max:np.float64(9.012866754084826) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3422235102189006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.012866754084826) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1422 - timing_s/gen:14.940290039405227 - timing_s/reward:0.05983664467930794 - timing_s/old_log_prob:2.2800443712621927 - timing_s/adv:0.5855069477111101 - timing_s/update_actor:15.32030195184052 - timing_s/step:33.273349760100245 - timing_s/testing:83.49365962296724 - timing_s/save_checkpoint:6.64394349232316 - timing_s/stop_profile:0.0001235567033290863 - timing_per_token_ms/update_actor:0.09218712626042061 - timing_per_token_ms/adv:0.0035231814023426026 - timing_per_token_ms/gen:0.1536260813709394 - perf/total_num_tokens:166187 - perf/time_per_step:33.273349760100245 - perf/throughput:624.3247268392076 (TaskRunner pid=2074004) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2074004) "np.float64(0.7109375), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2074004) "np.float64(0.16604232971456567), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2074004) "np.float64(0.78), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2074004) "np.float64(0.1094670184264562), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2074004) "np.float64(0.645725), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2074004) "np.float64(0.16903151272091094), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2074004) "np.float64(0.7134375000000001), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2074004) "np.float64(0.16525033629837935), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2074004) "np.float64(0.8191750000000001), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2074004) "np.float64(0.0603992963721259), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2074004) "np.float64(0.57575), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2074004) "np.float64(0.15001400456735414), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2074004) "np.float64(0.733725), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2074004) "np.float64(0.1297479179197353), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2074004) "np.float64(0.8390250000000001), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2074004) "np.float64(0.02670945685411765), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2074004) "np.float64(0.5126625), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2074004) "np.float64(0.12034852442580153), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2074004) "np.float64(0.7460875), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2074004) "np.float64(0.0934240016452198), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2074004) "np.float64(0.8469249999999999), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2074004) 'np.float64(0.009961583262996359), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/mean@16': np.float64(0.7109375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/std@16': np.float64(0.16604232971456567), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.78), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1094670184264562), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.645725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.16903151272091094), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.7134375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.16525033629837935), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.8191750000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.0603992963721259), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.57575), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.15001400456735414), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.733725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.1297479179197353), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8390250000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.02670945685411765), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.5126625), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.12034852442580153), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7460875), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.0934240016452198), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8469249999999999), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.009961583262996359), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/mean@16': np.float64(0.7109375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/std@16': np.float64(0.16604232971456567), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.78), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1094670184264562), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.645725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.16903151272091094), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.7134375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.16525033629837935), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.8191750000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.0603992963721259), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.57575), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.15001400456735414), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.733725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.1297479179197353), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8390250000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.02670945685411765), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.5126625), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.12034852442580153), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7460875), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.0934240016452198), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8469249999999999), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.009961583262996359), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9984375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2074004) 'np.float64(0.006051536478449089), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.9999125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2074004) 'np.float64(0.001440128696321335), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2074004) 'np.float64(0.9969625000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2074004) 'np.float64(0.008153644123697198), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9985125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2074004) 'np.float64(0.005913422059734724), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2074004) 'np.float64(0.9942624999999999), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2074004) 'np.float64(0.010511931298848583), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.9997125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2074004) 'np.float64(0.002649136032174741), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2074004) 'np.float64(0.9901125000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2074004) 'np.float64(0.012221532639586832), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2074004) 'np.float64(0.9999874999999999), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2074004) 'np.float64(0.00039508701573197775), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2074004) 'np.float64(0.9841000000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2074004) 'np.float64(0.012028687586990721), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2074004) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2074004) Training Progress: 100%|██████████| 100/100 [1:23:46<00:00, 62.98s/it] Training Progress: 100%|██████████| 100/100 [1:23:46<00:00, 50.26s/it] (TaskRunner pid=2074004) wandb: updating run metadata (TaskRunner pid=2074004) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2074004) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2074004) wandb: uploading history steps 98-99, summary, console lines 791-912 (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: Run history: (TaskRunner pid=2074004) wandb: actor/entropy ▇▆█▇▇█▅▆▄▃▂▆▃▂▄▁▄▃▂▁▂▁▃▄▄▅▃▅▃▄▆▄▄▆▄▄▅▆▃▄ (TaskRunner pid=2074004) wandb: actor/grad_norm ▇▃█▂▃▃▅▄▅▄▃▂▄▃▃▃▄▄▂▃▃▃▂▃▃▂▂▃▁▂▃▂▂▂▂▃▂▄▁▁ (TaskRunner pid=2074004) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2074004) wandb: actor/lr ▁▂▃█████████████████████████████████████ (TaskRunner pid=2074004) wandb: actor/pg_clipfrac ▁▄▄▅▄▄▄▄▅▇▅▆▄▄▄▆▄▆▃▃▅▅▅▆▅▄▆▅▄▃▂▂▅▃▄▂█▆▄▄ (TaskRunner pid=2074004) wandb: actor/pg_clipfrac_lower ▁▅▁▁▁▁▁█▅▁▁▁▃▁▆▁▁▁▁▃▃▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2074004) wandb: actor/pg_loss ▄▇█▄▇▅▇▇▅▅▄▆▆▅▄▄▆▄▄▆▄▃▂▃▄▃▂▂▃▃▄▄▄▃▅▁▃▃▅▂ (TaskRunner pid=2074004) wandb: actor/ppo_kl ▁▄▂▂▃▇▂▂▄▂▄▄▃▃▃▅▃▄▃▄▇▃▃▄▃▄▃▄▃█▃▃▅▄▄▃▄▃▄▃ (TaskRunner pid=2074004) wandb: critic/advantages/max ▇███▇████████▇██▅█▅▇██▇███▁██▇▇████▇███▇ (TaskRunner pid=2074004) wandb: critic/advantages/mean ▇▃▃▅▄▃▅▆▇▆▅█▃▆▃▃▃▄▃▃▅▆▇▄▁▇▆▇▃▅▅▅▇▇▆▅▅▄▅▅ (TaskRunner pid=2074004) wandb: +220 ... (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: Run summary: (TaskRunner pid=2074004) wandb: actor/entropy 0.10675 (TaskRunner pid=2074004) wandb: actor/grad_norm 0.18284 (TaskRunner pid=2074004) wandb: actor/kl_loss 0 (TaskRunner pid=2074004) wandb: actor/lr 0.0 (TaskRunner pid=2074004) wandb: actor/pg_clipfrac 0.00054 (TaskRunner pid=2074004) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2074004) wandb: actor/pg_loss 0.00128 (TaskRunner pid=2074004) wandb: actor/ppo_kl 2e-05 (TaskRunner pid=2074004) wandb: critic/advantages/max 0.875 (TaskRunner pid=2074004) wandb: critic/advantages/mean -0.00313 (TaskRunner pid=2074004) wandb: +220 ... (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: 🚀 View run qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/eajdt9hv (TaskRunner pid=2074004) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2074004) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2074004) wandb: Find logs at: ./wandb/run-20260702_085822-eajdt9hv/logs (TaskRunner pid=2074004) Exception ignored in atexit callback: .teardown_atexit at 0x7f1bd03c7880> (TaskRunner pid=2074004) Traceback (most recent call last): (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2074004) conn.teardown(hooks.exit_code) (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2074004) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2074004) return future.result() (TaskRunner pid=2074004) ^^^^^^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2074004) return self.__get_result() (TaskRunner pid=2074004) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2074004) raise self._exception (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2074004) return await fn() (TaskRunner pid=2074004) ^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2074004) await self._client.close() (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2074004) await self._writer.wait_closed() (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2074004) await self._protocol._get_close_waiter(self) (TaskRunner pid=2074004) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 10:22:16] Finished physics rlsd_tr [2026-07-02 10:22:16] Starting biology grpo Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_grpo_Qwen3_4B 2026-07-02 10:22:17,959 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,961 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,961 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,962 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,970 VINFO scripts.py:1286 -- Killed `/mnt/mole/SDPO/L2T/.venv/bin/python -u /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/runtime_env/agent/main.py --node-ip-address=198.19.44.212 --runtime-env-agent-port=56317 --gcs-address=198.19.44.212:57594 --cluster-id-hex=1a1f12b856019ab7823cf8c9c76e73dc7eaa5439177dd16f1d3c0b76 --runtime-env-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/runtime_resources --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --log-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs --temp-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B --stdout-filepath=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs/runtime_env_agent.out --stderr-filepath=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs/runtime_env_agent.err` (via SIGKILL) 2026-07-02 10:22:25,626 WARN scripts.py:1369 -- Stopped only 0 out of 8 Ray processes within the grace period 16 seconds. Set `-v` to see more details. Remaining processes [psutil.Process(pid=2074225, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072135, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072138, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072091, name='python', status='zombie'), psutil.Process(pid=2072147, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072134, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072141, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072137, name='ray::IDLE', status='zombie')] will be forcefully terminated. 2026-07-02 10:22:25,626 WARN scripts.py:1376 -- You can also use `--force` to forcefully terminate processes or set higher `--grace-period` to wait longer time for proper termination. Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 10:22:36,329 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2099163) TaskRunner hostname: mole-workspace-rw, PID: 2099163 (TaskRunner pid=2099163) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2099163) 'calculate_entropy': False, (TaskRunner pid=2099163) 'calculate_sum_pi_squared': False, (TaskRunner pid=2099163) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2099163) 'async_save': False, (TaskRunner pid=2099163) 'load_contents': ['model', (TaskRunner pid=2099163) 'optimizer', (TaskRunner pid=2099163) 'extra'], (TaskRunner pid=2099163) 'save_contents': ['model', (TaskRunner pid=2099163) 'optimizer', (TaskRunner pid=2099163) 'extra']}, (TaskRunner pid=2099163) 'clip_ratio': 0.2, (TaskRunner pid=2099163) 'clip_ratio_c': 3.0, (TaskRunner pid=2099163) 'clip_ratio_high': 0.28, (TaskRunner pid=2099163) 'clip_ratio_low': 0.2, (TaskRunner pid=2099163) 'data_loader_seed': 42, (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_coeff': 0, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'freeze_vision_tower': False, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': False, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'grad_clip': 1.0, (TaskRunner pid=2099163) 'kl_loss_coef': 0.0, (TaskRunner pid=2099163) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2099163) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2099163) 'loss_scale_factor': None, (TaskRunner pid=2099163) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2099163) 'betas': [0.9, 0.999], (TaskRunner pid=2099163) 'clip_grad': 1.0, (TaskRunner pid=2099163) 'lr': 1e-06, (TaskRunner pid=2099163) 'lr_scheduler_type': 'constant', (TaskRunner pid=2099163) 'lr_warmup_steps': 10, (TaskRunner pid=2099163) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2099163) 'min_lr_ratio': 0.0, (TaskRunner pid=2099163) 'num_cycles': 0.5, (TaskRunner pid=2099163) 'optimizer': 'AdamW', (TaskRunner pid=2099163) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2099163) 'override_optimizer_config': None, (TaskRunner pid=2099163) 'total_training_steps': -1, (TaskRunner pid=2099163) 'warmup_style': None, (TaskRunner pid=2099163) 'weight_decay': 0.01}, (TaskRunner pid=2099163) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2099163) 'clip_cov_lb': 1.0, (TaskRunner pid=2099163) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2099163) 'clip_cov_ub': 5.0, (TaskRunner pid=2099163) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2099163) 'loss_mode': 'vanilla', (TaskRunner pid=2099163) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2099163) 'ppo_epochs': 1, (TaskRunner pid=2099163) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'ppo_micro_batch_size': None, (TaskRunner pid=2099163) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'ppo_mini_batch_size': 8, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2099163) 'mode': 'disabled', (TaskRunner pid=2099163) 'record_file': None, (TaskRunner pid=2099163) 'replay_file': None}, (TaskRunner pid=2099163) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2099163) 'alpha': 0.5, (TaskRunner pid=2099163) 'distillation_add_tail': True, (TaskRunner pid=2099163) 'distillation_topk': 100, (TaskRunner pid=2099163) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2099163) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2099163) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'loss_weight': 0.0, (TaskRunner pid=2099163) 'mask': 'all'}, (TaskRunner pid=2099163) 'feedback_template': '\n' (TaskRunner pid=2099163) 'The ' (TaskRunner pid=2099163) 'following ' (TaskRunner pid=2099163) 'is ' (TaskRunner pid=2099163) 'feedback ' (TaskRunner pid=2099163) 'from ' (TaskRunner pid=2099163) 'your ' (TaskRunner pid=2099163) 'unsuccessful ' (TaskRunner pid=2099163) 'earlier ' (TaskRunner pid=2099163) 'attempt:\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) '{feedback_raw}', (TaskRunner pid=2099163) 'full_logit_distillation': True, (TaskRunner pid=2099163) 'include_environment_feedback': True, (TaskRunner pid=2099163) 'is_clip': 2, (TaskRunner pid=2099163) 'max_reprompt_len': 22528, (TaskRunner pid=2099163) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2099163) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) 'Correctly ' (TaskRunner pid=2099163) 'solve ' (TaskRunner pid=2099163) 'the ' (TaskRunner pid=2099163) 'original ' (TaskRunner pid=2099163) 'question.', (TaskRunner pid=2099163) 'reprompt_truncation': 'right', (TaskRunner pid=2099163) 'solution_template': '\n' (TaskRunner pid=2099163) 'Correct ' (TaskRunner pid=2099163) 'solution:\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) '{successful_previous_attempt}', (TaskRunner pid=2099163) 'srpo_dynamic_weighting': False, (TaskRunner pid=2099163) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2099163) 'success_reward_threshold': 0.5, (TaskRunner pid=2099163) 'teacher_regularization': 'ema', (TaskRunner pid=2099163) 'teacher_update_rate': 0.0, (TaskRunner pid=2099163) 'token_reweight_decay_steps': None, (TaskRunner pid=2099163) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2099163) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2099163) 'shuffle': False, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2099163) 'tau_neg': 1.05, (TaskRunner pid=2099163) 'tau_pos': 1.0, (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_kl_loss': False, (TaskRunner pid=2099163) 'use_prefix_grouper': False, (TaskRunner pid=2099163) 'use_remove_padding': True, (TaskRunner pid=2099163) 'use_torch_compile': True}, (TaskRunner pid=2099163) 'hybrid_engine': True, (TaskRunner pid=2099163) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2099163) 'custom_chat_template': None, (TaskRunner pid=2099163) 'enable_activation_offload': False, (TaskRunner pid=2099163) 'enable_gradient_checkpointing': True, (TaskRunner pid=2099163) 'exclude_modules': None, (TaskRunner pid=2099163) 'external_lib': None, (TaskRunner pid=2099163) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2099163) 'hf_config_path': None, (TaskRunner pid=2099163) 'lora_adapter_path': None, (TaskRunner pid=2099163) 'lora_alpha': 16, (TaskRunner pid=2099163) 'lora_rank': 0, (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'target_modules': 'all-linear', (TaskRunner pid=2099163) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2099163) 'num_shards': 4}, (TaskRunner pid=2099163) 'tokenizer_path': None, (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_liger': False, (TaskRunner pid=2099163) 'use_remove_padding': True, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'nccl_timeout': 600, (TaskRunner pid=2099163) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': True, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'log_prob_micro_batch_size': None, (TaskRunner pid=2099163) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2099163) 'mode': 'disabled', (TaskRunner pid=2099163) 'record_file': None, (TaskRunner pid=2099163) 'replay_file': None}, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_torch_compile': True}, (TaskRunner pid=2099163) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2099163) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2099163) 'agent_loop_config_path': None, (TaskRunner pid=2099163) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2099163) 'name': None, (TaskRunner pid=2099163) 'path': None}, (TaskRunner pid=2099163) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2099163) 'num_workers': 8}, (TaskRunner pid=2099163) 'calculate_log_probs': True, (TaskRunner pid=2099163) 'cudagraph_capture_sizes': None, (TaskRunner pid=2099163) 'data_parallel_size': 1, (TaskRunner pid=2099163) 'disable_log_stats': True, (TaskRunner pid=2099163) 'do_sample': True, (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'enable_chunked_prefill': True, (TaskRunner pid=2099163) 'enable_prefix_caching': True, (TaskRunner pid=2099163) 'enable_rollout_routing_replay': False, (TaskRunner pid=2099163) 'enforce_eager': False, (TaskRunner pid=2099163) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2099163) 'expert_parallel_size': 1, (TaskRunner pid=2099163) 'free_cache_engine': True, (TaskRunner pid=2099163) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2099163) 'ignore_eos': False, (TaskRunner pid=2099163) 'layered_summon': False, (TaskRunner pid=2099163) 'load_format': 'dummy', (TaskRunner pid=2099163) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'log_prob_micro_batch_size': None, (TaskRunner pid=2099163) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2099163) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2099163) 'max_model_len': 10240, (TaskRunner pid=2099163) 'max_num_batched_tokens': 10240, (TaskRunner pid=2099163) 'max_num_seqs': 1024, (TaskRunner pid=2099163) 'mode': 'async', (TaskRunner pid=2099163) 'multi_stage_wake_up': False, (TaskRunner pid=2099163) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'format': 'hermes', (TaskRunner pid=2099163) 'interaction_config_path': None, (TaskRunner pid=2099163) 'max_assistant_turns': None, (TaskRunner pid=2099163) 'max_parallel_calls': 1, (TaskRunner pid=2099163) 'max_tool_response_length': 256, (TaskRunner pid=2099163) 'max_user_turns': None, (TaskRunner pid=2099163) 'num_repeat_rollouts': None, (TaskRunner pid=2099163) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2099163) 'tool_config_path': None, (TaskRunner pid=2099163) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2099163) 'use_inference_chat_template': False}, (TaskRunner pid=2099163) 'n': 8, (TaskRunner pid=2099163) 'name': 'vllm', (TaskRunner pid=2099163) 'over_sample_rate': 0, (TaskRunner pid=2099163) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2099163) 'port': 9090, (TaskRunner pid=2099163) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2099163) 'prompt_length': 2048, (TaskRunner pid=2099163) 'quantization': None, (TaskRunner pid=2099163) 'quantization_config_file': None, (TaskRunner pid=2099163) 'response_length': 8192, (TaskRunner pid=2099163) 'scheduling_policy': 'fcfs', (TaskRunner pid=2099163) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2099163) 'skip_rollout': False, (TaskRunner pid=2099163) 'skip_tokenizer_init': True, (TaskRunner pid=2099163) 'temperature': 1.0, (TaskRunner pid=2099163) 'tensor_model_parallel_size': 2, (TaskRunner pid=2099163) 'top_k': -1, (TaskRunner pid=2099163) 'top_p': 1.0, (TaskRunner pid=2099163) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2099163) 'backend': None, (TaskRunner pid=2099163) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2099163) 'token2text': False}, (TaskRunner pid=2099163) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2099163) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2099163) 'do_sample': True, (TaskRunner pid=2099163) 'n': 16, (TaskRunner pid=2099163) 'temperature': 0.6, (TaskRunner pid=2099163) 'top_k': -1, (TaskRunner pid=2099163) 'top_p': 0.95}}}, (TaskRunner pid=2099163) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2099163) 'adv_estimator': 'grpo', (TaskRunner pid=2099163) 'gamma': 1.0, (TaskRunner pid=2099163) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2099163) 'horizon': 10000, (TaskRunner pid=2099163) 'kl_coef': 0.001, (TaskRunner pid=2099163) 'target_kl': 0.1, (TaskRunner pid=2099163) 'type': 'fixed'}, (TaskRunner pid=2099163) 'kl_penalty': 'kl', (TaskRunner pid=2099163) 'lam': 1.0, (TaskRunner pid=2099163) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2099163) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2099163) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2099163) 'loss_type': 'ppo_clip', (TaskRunner pid=2099163) 'rollout_is': 'token', (TaskRunner pid=2099163) 'rollout_is_batch_normalize': False, (TaskRunner pid=2099163) 'rollout_is_threshold': 2.0, (TaskRunner pid=2099163) 'rollout_rs': None, (TaskRunner pid=2099163) 'rollout_rs_threshold': None}, (TaskRunner pid=2099163) 'use_kl_in_reward': False, (TaskRunner pid=2099163) 'use_pf_ppo': False}, (TaskRunner pid=2099163) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2099163) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2099163) 'async_save': False, (TaskRunner pid=2099163) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2099163) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2099163) 'cliprange_value': 0.5, (TaskRunner pid=2099163) 'data_loader_seed': 42, (TaskRunner pid=2099163) 'enable': None, (TaskRunner pid=2099163) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'forward_micro_batch_size': None, (TaskRunner pid=2099163) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'grad_clip': 1.0, (TaskRunner pid=2099163) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2099163) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2099163) 'enable_activation_offload': False, (TaskRunner pid=2099163) 'enable_gradient_checkpointing': True, (TaskRunner pid=2099163) 'external_lib': None, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': False, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'lora_alpha': 16, (TaskRunner pid=2099163) 'lora_rank': 0, (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2099163) 'target_modules': 'all-linear', (TaskRunner pid=2099163) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2099163) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_remove_padding': False, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2099163) 'betas': [0.9, 0.999], (TaskRunner pid=2099163) 'clip_grad': 1.0, (TaskRunner pid=2099163) 'lr': 1e-05, (TaskRunner pid=2099163) 'lr_scheduler_type': 'constant', (TaskRunner pid=2099163) 'lr_warmup_steps': -1, (TaskRunner pid=2099163) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2099163) 'min_lr_ratio': 0.0, (TaskRunner pid=2099163) 'num_cycles': 0.5, (TaskRunner pid=2099163) 'optimizer': 'AdamW', (TaskRunner pid=2099163) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2099163) 'override_optimizer_config': None, (TaskRunner pid=2099163) 'total_training_steps': -1, (TaskRunner pid=2099163) 'warmup_style': None, (TaskRunner pid=2099163) 'weight_decay': 0.01}, (TaskRunner pid=2099163) 'ppo_epochs': 1, (TaskRunner pid=2099163) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'ppo_micro_batch_size': None, (TaskRunner pid=2099163) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'ppo_mini_batch_size': 8, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'shuffle': False, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False}, (TaskRunner pid=2099163) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2099163) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2099163) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2099163) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2099163) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2099163) 'dataloader_num_workers': 8, (TaskRunner pid=2099163) 'filter_overlong_prompts': True, (TaskRunner pid=2099163) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2099163) 'image_key': 'images', (TaskRunner pid=2099163) 'image_patch_size': 14, (TaskRunner pid=2099163) 'max_prompt_length': 2048, (TaskRunner pid=2099163) 'max_response_length': 8192, (TaskRunner pid=2099163) 'prompt_key': 'prompt', (TaskRunner pid=2099163) 'return_full_prompt': False, (TaskRunner pid=2099163) 'return_multi_modal_inputs': True, (TaskRunner pid=2099163) 'return_raw_chat': True, (TaskRunner pid=2099163) 'return_raw_input_ids': False, (TaskRunner pid=2099163) 'reward_fn_key': 'data_source', (TaskRunner pid=2099163) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2099163) 'seed': None, (TaskRunner pid=2099163) 'shuffle': True, (TaskRunner pid=2099163) 'tokenizer': None, (TaskRunner pid=2099163) 'tool_config_path': None, (TaskRunner pid=2099163) 'train_batch_size': 32, (TaskRunner pid=2099163) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2099163) 'train_max_samples': 3200, (TaskRunner pid=2099163) 'truncation': 'error', (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_shm': False, (TaskRunner pid=2099163) 'val_batch_size': None, (TaskRunner pid=2099163) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2099163) 'val_max_samples': -1, (TaskRunner pid=2099163) 'validation_shuffle': False, (TaskRunner pid=2099163) 'video_key': 'videos'}, (TaskRunner pid=2099163) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2099163) 'cuda-memory-usage': 'true', (TaskRunner pid=2099163) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2099163) 'capture-range-end': None, (TaskRunner pid=2099163) 'cuda-graph-trace': 'graph', (TaskRunner pid=2099163) 'cuda-memory-usage': 'true', (TaskRunner pid=2099163) 'kill': 'none', (TaskRunner pid=2099163) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2099163) 'torch_memory': {'context': 'all', (TaskRunner pid=2099163) 'kw_args': {}, (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'stacks': 'all', (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2099163) 'profile_continuous_steps': False, (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'steps': None, (TaskRunner pid=2099163) 'tool': None}, (TaskRunner pid=2099163) 'max_model_len': 10240, (TaskRunner pid=2099163) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_4B', (TaskRunner pid=2099163) 'include_dashboard': False, (TaskRunner pid=2099163) 'num_cpus': None, (TaskRunner pid=2099163) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2099163) 'USER': 'root'}}}, (TaskRunner pid=2099163) 'timeline_json_file': None}, (TaskRunner pid=2099163) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2099163) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2099163) 'name': 'custom_reward_manager', (TaskRunner pid=2099163) 'path': None}, (TaskRunner pid=2099163) 'name': 'naive', (TaskRunner pid=2099163) 'source': 'register'}, (TaskRunner pid=2099163) 'reward_model': {'enable': False, (TaskRunner pid=2099163) 'enable_resource_pool': False, (TaskRunner pid=2099163) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'launch_reward_fn_async': False, (TaskRunner pid=2099163) 'max_length': None, (TaskRunner pid=2099163) 'micro_batch_size': None, (TaskRunner pid=2099163) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'model': {'external_lib': None, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2099163) 'trust_remote_code': False, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_remove_padding': False, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'n_gpus_per_node': 8, (TaskRunner pid=2099163) 'nnodes': 0, (TaskRunner pid=2099163) 'num_workers': 1, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'reward_loop_class_name': None, (TaskRunner pid=2099163) 'reward_loop_module_path': None, (TaskRunner pid=2099163) 'reward_loop_source': 'register', (TaskRunner pid=2099163) 'reward_manager': 'naive', (TaskRunner pid=2099163) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2099163) 'cudagraph_capture_sizes': None, (TaskRunner pid=2099163) 'data_parallel_size': 1, (TaskRunner pid=2099163) 'disable_log_stats': True, (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'enable_chunked_prefill': True, (TaskRunner pid=2099163) 'enable_prefix_caching': True, (TaskRunner pid=2099163) 'enforce_eager': True, (TaskRunner pid=2099163) 'engine_kwargs': {}, (TaskRunner pid=2099163) 'expert_parallel_size': 1, (TaskRunner pid=2099163) 'free_cache_engine': True, (TaskRunner pid=2099163) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2099163) 'limit_images': None, (TaskRunner pid=2099163) 'load_format': 'auto', (TaskRunner pid=2099163) 'max_model_len': None, (TaskRunner pid=2099163) 'max_num_batched_tokens': 8192, (TaskRunner pid=2099163) 'max_num_seqs': 1024, (TaskRunner pid=2099163) 'name': '???', (TaskRunner pid=2099163) 'prompt_length': 2048, (TaskRunner pid=2099163) 'response_length': 2048, (TaskRunner pid=2099163) 'skip_tokenizer_init': False, (TaskRunner pid=2099163) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2099163) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2099163) 'memory_limit_mb': 1024, (TaskRunner pid=2099163) 'url': None}, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False, (TaskRunner pid=2099163) 'use_reward_loop': False}, (TaskRunner pid=2099163) 'trainer': {'balance_batch': True, (TaskRunner pid=2099163) 'critic_warmup': 0, (TaskRunner pid=2099163) 'default_hdfs_dir': None, (TaskRunner pid=2099163) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'del_local_ckpt_after_load': False, (TaskRunner pid=2099163) 'device': 'cuda', (TaskRunner pid=2099163) 'esi_redundant_time': 0, (TaskRunner pid=2099163) 'experiment_name': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2099163) 'log_val_generations': 0, (TaskRunner pid=2099163) 'logger': ['console', 'wandb'], (TaskRunner pid=2099163) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2099163) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2099163) 'n_gpus_per_node': 8, (TaskRunner pid=2099163) 'nnodes': 1, (TaskRunner pid=2099163) 'project_name': 'SDPO-root', (TaskRunner pid=2099163) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2099163) 'resume_from_path': None, (TaskRunner pid=2099163) 'resume_mode': 'auto', (TaskRunner pid=2099163) 'rollout_data_dir': None, (TaskRunner pid=2099163) 'save_freq': 10, (TaskRunner pid=2099163) 'test_freq': 10, (TaskRunner pid=2099163) 'total_epochs': 30, (TaskRunner pid=2099163) 'total_training_steps': 100, (TaskRunner pid=2099163) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2099163) 'val_before_train': False, (TaskRunner pid=2099163) 'val_only': False, (TaskRunner pid=2099163) 'validation_data_dir': None}, (TaskRunner pid=2099163) 'transfer_queue': {'enable': False}, (TaskRunner pid=2099163) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2099163) 'dir': '/users/root/SDPO', (TaskRunner pid=2099163) 'log_dir': '/users/root/output', (TaskRunner pid=2099163) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2099163) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2099163) use_critic=need_critic(config), (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2099163) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2099163) Using dataset class: RLHFDataset (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2099163) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2099163) dataset len: 450 (TaskRunner pid=2099163) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2099163) WARNING:2026-07-02 10:22:45,455:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2099163) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2099163) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2099163) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 50/50 [00:00<00:00, 79.25 examples/s] (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2099163) self.use_critic = need_critic(self.config) (WorkerDict pid=2099540) [W702 10:22:53.887126096 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53015 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2099539) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2099546) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2099546) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2099543) [W702 10:22:54.133676817 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53015 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2099539) Model config after override: Qwen3Config { (WorkerDict pid=2099539) "architectures": [ (WorkerDict pid=2099539) "Qwen3ForCausalLM" (WorkerDict pid=2099539) ], (WorkerDict pid=2099539) "attention_bias": false, (WorkerDict pid=2099539) "attention_dropout": 0.0, (WorkerDict pid=2099539) "dtype": "bfloat16", (WorkerDict pid=2099539) "eos_token_id": 151645, (WorkerDict pid=2099539) "head_dim": 128, (WorkerDict pid=2099539) "hidden_act": "silu", (WorkerDict pid=2099539) "hidden_size": 2560, (WorkerDict pid=2099539) "initializer_range": 0.02, (WorkerDict pid=2099539) "intermediate_size": 9728, (WorkerDict pid=2099539) "layer_types": [ (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention" (WorkerDict pid=2099539) ], (WorkerDict pid=2099539) "max_position_embeddings": 40960, (WorkerDict pid=2099539) "max_window_layers": 36, (WorkerDict pid=2099539) "model_type": "qwen3", (WorkerDict pid=2099539) "num_attention_heads": 32, (WorkerDict pid=2099539) "num_hidden_layers": 36, (WorkerDict pid=2099539) "num_key_value_heads": 8, (WorkerDict pid=2099539) "pad_token_id": 151643, (WorkerDict pid=2099539) "rms_norm_eps": 1e-06, (WorkerDict pid=2099539) "rope_scaling": null, (WorkerDict pid=2099539) "rope_theta": 1000000, (WorkerDict pid=2099539) "sliding_window": null, (WorkerDict pid=2099539) "tie_word_embeddings": true, (WorkerDict pid=2099539) "transformers_version": "4.57.1", (WorkerDict pid=2099539) "use_cache": true, (WorkerDict pid=2099539) "use_sliding_window": false, (WorkerDict pid=2099539) "vocab_size": 151936 (WorkerDict pid=2099539) } (WorkerDict pid=2099539) (WorkerDict pid=2099540) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2099540) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2099540) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2099546) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2099539) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2099540) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2099540) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2099539) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.93s/it] [repeated 14x across cluster] (WorkerDict pid=2099539) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 6x across cluster] (WorkerDict pid=2099539) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2099539) Actor use_remove_padding=True (WorkerDict pid=2099539) Actor use_fused_kernels=False (WorkerDict pid=2099539) Actor use_prefix_grouper=False (WorkerDict pid=2099543) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2099543) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2099540) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2099540) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2099539) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2099544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2099544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2099542) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2099542) warnings.warn( (WorkerDict pid=2099544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2099544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2099163) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2099543) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2099543) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2100408) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2100408) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2100407) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2100407) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2100407) ['serve', (vLLMHttpServer pid=2100407) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2100407) '--dtype', (vLLMHttpServer pid=2100407) 'bfloat16', (vLLMHttpServer pid=2100407) '--load_format', (vLLMHttpServer pid=2100407) 'dummy', (vLLMHttpServer pid=2100407) '--trust_remote_code', (vLLMHttpServer pid=2100407) '--max_model_len', (vLLMHttpServer pid=2100407) '40960', (vLLMHttpServer pid=2100407) '--max_num_seqs', (vLLMHttpServer pid=2100407) '1024', (vLLMHttpServer pid=2100407) '--enable_chunked_prefill', (vLLMHttpServer pid=2100407) '--max_num_batched_tokens', (vLLMHttpServer pid=2100407) '10240', (vLLMHttpServer pid=2100407) '--enable_prefix_caching', (vLLMHttpServer pid=2100407) '--enable_sleep_mode', (vLLMHttpServer pid=2100407) '--logprobs_mode', (vLLMHttpServer pid=2100407) 'processed_logprobs', (vLLMHttpServer pid=2100407) '--disable_custom_all_reduce', (vLLMHttpServer pid=2100407) '--gpu_memory_utilization', (vLLMHttpServer pid=2100407) '0.8', (vLLMHttpServer pid=2100407) '--disable_log_stats', (vLLMHttpServer pid=2100407) '--tensor_parallel_size', (vLLMHttpServer pid=2100407) '2', (vLLMHttpServer pid=2100407) '--seed', (vLLMHttpServer pid=2100407) '0', (vLLMHttpServer pid=2100407) '--override_generation_config', (vLLMHttpServer pid=2100407) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2100407) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2100407) '--hf_overrides', (vLLMHttpServer pid=2100407) '{}', (vLLMHttpServer pid=2100407) '--scheduling_policy', (vLLMHttpServer pid=2100407) 'fcfs'] (WorkerDict pid=2099544) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2100408) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2100408) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2100407) WARNING 07-02 10:23:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2099540) WARNING 07-02 10:23:47 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2100409) WARNING 07-02 10:23:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2099541) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2099541.136359408498368.tmp' (vLLMHttpServer pid=2100410) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2100410) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2100410) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2100410) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2099539) 2026-07-02 10:24:01,345 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2099539) 2026-07-02 10:24:01,370 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2099539) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence is typically determined through computational methods such as energy minimization, molecular dynamics simulations, or machine learning models trained on known protein structures. However, without access to specific computational tools or databases that can calculate the folding stability score for the given sequence, it is not possible to determine the exact score. The provided options (A: -0.01, B: 1.69, C: 2.49, D: 0.45) are likely hypothetical values, and there is no standard or universally accepted method to calculate this score from the sequence alone. Therefore, the correct answer cannot be determined based on the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) B (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 1.0 (TaskRunner pid=2099163) [acc] 1.0 (TaskRunner pid=2099163) [pred] B (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2099163) step:10 - global_seqlen/min:12666 - global_seqlen/max:26901 - global_seqlen/minmax_diff:14235 - global_seqlen/balanced_min:18269 - global_seqlen/balanced_max:18382 - global_seqlen/mean:18293.25 - actor/entropy:0.192046120762825 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9020472168922424 - training/rollout_probs_diff_mean:0.00564963323995471 - training/rollout_probs_diff_std:0.0170443132519722 - training/rollout_actor_probs_pearson_corr:0.9967436790466309 - rollout_corr/rollout_is_mean:0.9999064207077026 - rollout_corr/rollout_is_ratio_fraction_high:6.567023228853941e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00027581496397033334 - rollout_corr/rollout_is_max:3.955087184906006 - rollout_corr/rollout_is_min:0.025797156617045403 - rollout_corr/rollout_is_std:0.045984067022800446 - rollout_corr/rollout_is_eff_sample_size:0.997889452616177 - rollout_corr/rollout_is_seq_mean:0.9998351335525513 - rollout_corr/rollout_is_seq_std:0.0035011477302759886 - rollout_corr/rollout_is_seq_max:1.0109554529190063 - rollout_corr/rollout_is_seq_min:0.9882628321647644 - rollout_corr/rollout_is_seq_max_deviation:0.011737167835235596 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2451128889806569) - rollout_corr/training_log_ppl:np.float64(0.21647670843231026) - rollout_corr/kl:np.float64(0.00146516180673828) - rollout_corr/k3_kl:np.float64(0.0013313775715459997) - rollout_corr/rollout_ppl:np.float64(1.2432878208346665) - rollout_corr/rollout_log_ppl:np.float64(0.21501154491852503) - rollout_corr/log_ppl_diff:np.float64(0.0014651635137852281) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028982661897316575) - rollout_corr/log_ppl_diff_max:np.float64(0.014495670795440674) - rollout_corr/log_ppl_diff_min:np.float64(-0.009721904993057251) - rollout_corr/ppl_ratio:np.float64(1.0014725218061358) - rollout_corr/chi2_token:np.float64(0.0023953597992658615) - rollout_corr/chi2_seq:np.float64(1.8521191228646785) - actor/pg_loss:np.float64(-0.0003558297175914049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015348545880442543) - actor/ppo_kl:np.float64(3.0528818129482715e-05) - actor/pg_clipfrac_lower:np.float64(1.3610627320304047e-05) - actor/grad_norm:np.float64(0.7854970991611481) - perf/mfu/actor:np.float64(0.052699151040128595) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(105.79932022094727) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.32625) - val-aux/sciknoweval/reward/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17799526365500928) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6341) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.044642921667641916) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.35164) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13534911056216117) - val-aux/sciknoweval/score/mean@16:np.float64(0.32625) - val-aux/sciknoweval/score/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17799526365500928) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6341) - val-aux/sciknoweval/score/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.044642921667641916) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.35164) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13534911056216117) - val-core/sciknoweval/acc/mean@16:np.float64(0.32625) - val-aux/sciknoweval/acc/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17799526365500928) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6341) - val-core/sciknoweval/acc/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.044642921667641916) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.35164) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13534911056216117) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.97875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.051687330456951015) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9963400000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.020813886045474284) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.96174) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06405299669111861) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9787599999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05137839781051554) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9997800000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003963742623570884) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.93396) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07265674213411831) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99216) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03188606302646686) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.90036) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06915111790505142) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99804) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.015647532793846276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.86874) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05102367714768366) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002157810766489876) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.25 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036463395226746798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036463395226746798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:297.4140625 - response_length/max:1569.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.4140625 - response_length_non_aborted/max:1569.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015241466462612152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.17083459533751) - timing_s/agent_loop/generate_sequences/max:np.float64(9.452028851956129) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0214375181894866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.452028851956129) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1569 - timing_s/gen:15.476865164935589 - timing_s/reward:0.045249899849295616 - timing_s/old_log_prob:2.511210959404707 - timing_s/adv:0.15661288052797318 - timing_s/update_actor:10.127284951508045 - timing_s/step:28.350071949884295 - timing_s/testing:84.79899363964796 - timing_s/save_checkpoint:6.5641801711171865 - timing_s/stop_profile:0.00011404044926166534 - timing_per_token_ms/gen:0.20327386016096546 - timing_per_token_ms/update_actor:0.06920096860527822 - timing_per_token_ms/adv:0.0010701548421410436 - perf/total_num_tokens:146346 - perf/time_per_step:28.350071949884295 - perf/throughput:645.2629126422609 (TaskRunner pid=2099163) Training Progress: 10%|█ | 10/100 [10:46<1:44:31, 69.68s/it] (TaskRunner pid=2099163) step:11 - global_seqlen/min:16676 - global_seqlen/max:36461 - global_seqlen/minmax_diff:19785 - global_seqlen/balanced_min:22290 - global_seqlen/balanced_max:27044 - global_seqlen/mean:22898.125 - actor/entropy:0.16072751581668854 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32411471009254456 - training/rollout_probs_diff_mean:0.0046199411153793335 - training/rollout_probs_diff_std:0.014347893185913563 - training/rollout_actor_probs_pearson_corr:0.9974071383476257 - rollout_corr/rollout_is_mean:0.9998364448547363 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011153358354931697 - rollout_corr/rollout_is_max:1.8010543584823608 - rollout_corr/rollout_is_min:0.18950599431991577 - rollout_corr/rollout_is_std:0.040931250900030136 - rollout_corr/rollout_is_eff_sample_size:0.9983268407518708 - rollout_corr/rollout_is_seq_mean:0.9997417330741882 - rollout_corr/rollout_is_seq_std:0.0031297144014388323 - rollout_corr/rollout_is_seq_max:1.0107784271240234 - rollout_corr/rollout_is_seq_min:0.9905034303665161 - rollout_corr/rollout_is_seq_max_deviation:0.010778427124023438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2281816471368074) - rollout_corr/training_log_ppl:np.float64(0.2024953516520327) - rollout_corr/kl:np.float64(0.001424252133454651) - rollout_corr/k3_kl:np.float64(0.0011623206726625313) - rollout_corr/rollout_ppl:np.float64(1.226403083652258) - rollout_corr/rollout_log_ppl:np.float64(0.20107109841774218) - rollout_corr/log_ppl_diff:np.float64(0.0014242532342905179) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025687814049888402) - rollout_corr/log_ppl_diff_max:np.float64(0.01213720440864563) - rollout_corr/log_ppl_diff_min:np.float64(-0.009074568748474121) - rollout_corr/ppl_ratio:np.float64(1.0014300209004432) - rollout_corr/chi2_token:np.float64(0.0017990793567150831) - rollout_corr/chi2_seq:np.float64(0.5897399557288736) - actor/pg_loss:np.float64(0.00016360939480364323) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016948245472576673) - actor/ppo_kl:np.float64(9.670323938681946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6953910440206528) - perf/mfu/actor:np.float64(0.06338107402915627) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(111.92782211303711) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03447655215859413 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03447655215859413 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:385.25390625 - response_length/max:6669.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:385.25390625 - response_length_non_aborted/max:6669.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.39831817150116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2415860313922167) - timing_s/agent_loop/generate_sequences/max:np.float64(33.86591459810734) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7203849007346435) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.86591459810734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:6669 - timing_s/gen:39.782698061317205 - timing_s/reward:0.061702121049165726 - timing_s/old_log_prob:2.763470709323883 - timing_s/adv:0.13837043195962906 - timing_s/update_actor:10.400962248444557 - timing_s/step:53.240667348727584 - timing_s/stop_profile:0.00013394653797149658 - timing_per_token_ms/gen:0.403373364373305 - timing_per_token_ms/update_actor:0.05677846029120592 - timing_per_token_ms/adv:0.0007553589647603737 - perf/total_num_tokens:183185 - perf/time_per_step:53.240667348727584 - perf/throughput:430.0871146114071 (TaskRunner pid=2099163) Training Progress: 11%|█ | 11/100 [11:39<1:35:55, 64.66s/it] (TaskRunner pid=2099163) step:12 - global_seqlen/min:20156 - global_seqlen/max:34994 - global_seqlen/minmax_diff:14838 - global_seqlen/balanced_min:26866 - global_seqlen/balanced_max:32121 - global_seqlen/mean:28280.875 - actor/entropy:0.12687404453754425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5569577217102051 - training/rollout_probs_diff_mean:0.00370349595323205 - training/rollout_probs_diff_std:0.013641891069710255 - training/rollout_actor_probs_pearson_corr:0.9971849918365479 - rollout_corr/rollout_is_mean:0.999925971031189 - rollout_corr/rollout_is_ratio_fraction_high:2.9888440622016788e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020174698147457093 - rollout_corr/rollout_is_max:2.8711585998535156 - rollout_corr/rollout_is_min:0.05395948514342308 - rollout_corr/rollout_is_std:0.038417838513851166 - rollout_corr/rollout_is_eff_sample_size:0.9985261260875207 - rollout_corr/rollout_is_seq_mean:0.9998483657836914 - rollout_corr/rollout_is_seq_std:0.002559172920882702 - rollout_corr/rollout_is_seq_max:1.0071240663528442 - rollout_corr/rollout_is_seq_min:0.9910182356834412 - rollout_corr/rollout_is_seq_max_deviation:0.008981764316558838 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2076720832847059) - rollout_corr/training_log_ppl:np.float64(0.1855281984044268) - rollout_corr/kl:np.float64(0.0015311053801809749) - rollout_corr/k3_kl:np.float64(0.0011974603096120973) - rollout_corr/rollout_ppl:np.float64(1.205766570288688) - rollout_corr/rollout_log_ppl:np.float64(0.1839970920154883) - rollout_corr/log_ppl_diff:np.float64(0.0015311063889384968) - rollout_corr/log_ppl_abs_diff:np.float64(0.002373620836806367) - rollout_corr/log_ppl_diff_max:np.float64(0.015954285860061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.005851656198501587) - rollout_corr/ppl_ratio:np.float64(1.0015365811996162) - rollout_corr/chi2_token:np.float64(0.0016909947153180838) - rollout_corr/chi2_seq:np.float64(0.3893565512262285) - actor/pg_loss:np.float64(0.0003301744582131505) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001968728217093485) - actor/ppo_kl:np.float64(0.0002511454556559656) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8092714995145798) - perf/mfu/actor:np.float64(0.07718086515597372) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.70772171020508) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.019058551639318466 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.019058551639318466 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:522.77734375 - response_length/max:8192.0 - response_length/min:106.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:522.77734375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.196065366268158e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2030387092381716) - timing_s/agent_loop/generate_sequences/max:np.float64(49.85608519241214) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7326578640859225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.85608519241214) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.627652468159795 - timing_s/reward:0.06745622679591179 - timing_s/old_log_prob:2.655902164056897 - timing_s/adv:0.12508256547152996 - timing_s/update_actor:10.585711555555463 - timing_s/step:69.14367071911693 - timing_s/stop_profile:0.00011152029037475586 - timing_per_token_ms/gen:0.41565595764927254 - timing_per_token_ms/update_actor:0.04678829578096268 - timing_per_token_ms/adv:0.0005528584488259732 - perf/total_num_tokens:226247 - perf/time_per_step:69.14367071911693 - perf/throughput:409.016106693058 (TaskRunner pid=2099163) Training Progress: 12%|█▏ | 12/100 [12:48<1:36:51, 66.04s/it] (TaskRunner pid=2099163) step:13 - global_seqlen/min:14490 - global_seqlen/max:27440 - global_seqlen/minmax_diff:12950 - global_seqlen/balanced_min:21991 - global_seqlen/balanced_max:22034 - global_seqlen/mean:22017.0 - actor/entropy:0.1795348972082138 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3799695372581482 - training/rollout_probs_diff_mean:0.004953271709382534 - training/rollout_probs_diff_std:0.014711440540850163 - training/rollout_actor_probs_pearson_corr:0.9974781274795532 - rollout_corr/rollout_is_mean:1.0001226663589478 - rollout_corr/rollout_is_ratio_fraction_high:4.09433341701515e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.165083661675453e-05 - rollout_corr/rollout_is_max:2.9943156242370605 - rollout_corr/rollout_is_min:0.36588054895401 - rollout_corr/rollout_is_std:0.04241306707262993 - rollout_corr/rollout_is_eff_sample_size:0.9982047182816606 - rollout_corr/rollout_is_seq_mean:1.0001139640808105 - rollout_corr/rollout_is_seq_std:0.003129767719656229 - rollout_corr/rollout_is_seq_max:1.0108983516693115 - rollout_corr/rollout_is_seq_min:0.9917095303535461 - rollout_corr/rollout_is_seq_max_deviation:0.010898351669311523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2477836473844945) - rollout_corr/training_log_ppl:np.float64(0.2179029462276958) - rollout_corr/kl:np.float64(0.0012722289181801294) - rollout_corr/k3_kl:np.float64(0.0011847673267197933) - rollout_corr/rollout_ppl:np.float64(1.2461294089443982) - rollout_corr/rollout_log_ppl:np.float64(0.2166307152074296) - rollout_corr/log_ppl_diff:np.float64(0.0012722310202661902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027390209725126624) - rollout_corr/log_ppl_diff_max:np.float64(0.01292330026626587) - rollout_corr/log_ppl_diff_min:np.float64(-0.009444236755371094) - rollout_corr/ppl_ratio:np.float64(1.0012791764456779) - rollout_corr/chi2_token:np.float64(0.002235463121905923) - rollout_corr/chi2_seq:np.float64(1.5766265538986772) - actor/pg_loss:np.float64(-0.0002433034824207425) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002133346634764166) - actor/ppo_kl:np.float64(0.00032201276727761297) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8120427280664444) - perf/mfu/actor:np.float64(0.06433296210788333) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.85671615600586) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007056328002363443 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007056328002363443 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:381.625 - response_length/max:1800.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.625 - response_length_non_aborted/max:1800.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013713166117668152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2604616954922676) - timing_s/agent_loop/generate_sequences/max:np.float64(11.544187435880303) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8230461675921106) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.544187435880303) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1800 - timing_s/gen:17.578441470861435 - timing_s/reward:0.061520595103502274 - timing_s/old_log_prob:2.475315697491169 - timing_s/adv:0.12054650858044624 - timing_s/update_actor:9.704637175425887 - timing_s/step:30.0266327932477 - timing_s/stop_profile:0.0001079123467206955 - timing_per_token_ms/gen:0.17993000195362588 - timing_per_token_ms/update_actor:0.05509740868093909 - timing_per_token_ms/adv:0.0006843944939163274 - perf/total_num_tokens:176136 - perf/time_per_step:30.0266327932477 - perf/throughput:733.2490509875325 (TaskRunner pid=2099163) Training Progress: 13%|█▎ | 13/100 [13:18<1:19:57, 55.14s/it] (TaskRunner pid=2099163) step:14 - global_seqlen/min:12976 - global_seqlen/max:34060 - global_seqlen/minmax_diff:21084 - global_seqlen/balanced_min:22596 - global_seqlen/balanced_max:22654 - global_seqlen/mean:22633.375 - actor/entropy:0.16276216506958008 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31926417350769043 - training/rollout_probs_diff_mean:0.004311886616051197 - training/rollout_probs_diff_std:0.013316148892045021 - training/rollout_actor_probs_pearson_corr:0.9977449178695679 - rollout_corr/rollout_is_mean:0.9999713897705078 - rollout_corr/rollout_is_ratio_fraction_high:3.8242746086325496e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.692480383208022e-05 - rollout_corr/rollout_is_max:2.2574832439422607 - rollout_corr/rollout_is_min:0.28150883316993713 - rollout_corr/rollout_is_std:0.040303174406290054 - rollout_corr/rollout_is_eff_sample_size:0.9983781696179648 - rollout_corr/rollout_is_seq_mean:0.9999383687973022 - rollout_corr/rollout_is_seq_std:0.0024390218313783407 - rollout_corr/rollout_is_seq_max:1.008014440536499 - rollout_corr/rollout_is_seq_min:0.9899135828018188 - rollout_corr/rollout_is_seq_max_deviation:0.010086417198181152 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2327516474761069) - rollout_corr/training_log_ppl:np.float64(0.20615991155500524) - rollout_corr/kl:np.float64(0.0009014457496316908) - rollout_corr/k3_kl:np.float64(0.0010392485548891273) - rollout_corr/rollout_ppl:np.float64(1.231638588476926) - rollout_corr/rollout_log_ppl:np.float64(0.2052584641496651) - rollout_corr/log_ppl_diff:np.float64(0.00090144740534015) - rollout_corr/log_ppl_abs_diff:np.float64(0.00216034529148601) - rollout_corr/log_ppl_diff_max:np.float64(0.010936662554740906) - rollout_corr/log_ppl_diff_min:np.float64(-0.008887723088264465) - rollout_corr/ppl_ratio:np.float64(1.0009056760463864) - rollout_corr/chi2_token:np.float64(0.002428637119010091) - rollout_corr/chi2_seq:np.float64(1.1035329343285412) - actor/pg_loss:np.float64(0.0004473176086321473) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011060538008678122) - actor/ppo_kl:np.float64(-0.0001549605085848782) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.575629249215126) - perf/mfu/actor:np.float64(0.0652755181152363) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(109.99267196655273) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010551412589848042 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010551412589848042 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.57421875 - response_length/max:2188.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.57421875 - response_length_non_aborted/max:2188.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.820424020290375e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0610912814736366) - timing_s/agent_loop/generate_sequences/max:np.float64(13.642793396487832) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8376170305346022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.642793396487832) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:2188 - timing_s/gen:19.731356034055352 - timing_s/reward:0.06039970554411411 - timing_s/old_log_prob:2.502957135438919 - timing_s/adv:0.12832614220678806 - timing_s/update_actor:9.91347792558372 - timing_s/step:32.42584792152047 - timing_s/stop_profile:0.00012178905308246613 - timing_per_token_ms/gen:0.18864530841871363 - timing_per_token_ms/update_actor:0.05475032957735932 - timing_per_token_ms/adv:0.0007087218665288985 - perf/total_num_tokens:181067 - perf/time_per_step:32.42584792152047 - perf/throughput:698.0041063160178 (TaskRunner pid=2099163) Training Progress: 14%|█▍ | 14/100 [13:51<1:09:13, 48.30s/it] (TaskRunner pid=2099163) step:15 - global_seqlen/min:15918 - global_seqlen/max:28170 - global_seqlen/minmax_diff:12252 - global_seqlen/balanced_min:19653 - global_seqlen/balanced_max:19679 - global_seqlen/mean:19673.25 - actor/entropy:0.19017662107944489 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6307171583175659 - training/rollout_probs_diff_mean:0.005055586341768503 - training/rollout_probs_diff_std:0.014726489782333374 - training/rollout_actor_probs_pearson_corr:0.9975889921188354 - rollout_corr/rollout_is_mean:1.0000306367874146 - rollout_corr/rollout_is_ratio_fraction_high:2.42359619733179e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013329778448678553 - rollout_corr/rollout_is_max:2.5952959060668945 - rollout_corr/rollout_is_min:0.22091145813465118 - rollout_corr/rollout_is_std:0.04240182042121887 - rollout_corr/rollout_is_eff_sample_size:0.9982053121902402 - rollout_corr/rollout_is_seq_mean:1.0000522136688232 - rollout_corr/rollout_is_seq_std:0.0030719421338289976 - rollout_corr/rollout_is_seq_max:1.0085277557373047 - rollout_corr/rollout_is_seq_min:0.9906306266784668 - rollout_corr/rollout_is_seq_max_deviation:0.009369373321533203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2559455754235387) - rollout_corr/training_log_ppl:np.float64(0.22441870825423393) - rollout_corr/kl:np.float64(0.001127970071593154) - rollout_corr/k3_kl:np.float64(0.001125821751145395) - rollout_corr/rollout_ppl:np.float64(1.254524188581854) - rollout_corr/rollout_log_ppl:np.float64(0.22329073723813053) - rollout_corr/log_ppl_diff:np.float64(0.0011279710161034018) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024534623080398887) - rollout_corr/log_ppl_diff_max:np.float64(0.014268159866333008) - rollout_corr/log_ppl_diff_min:np.float64(-0.008966684341430664) - rollout_corr/ppl_ratio:np.float64(1.00113360513933) - rollout_corr/chi2_token:np.float64(0.0023321015760302544) - rollout_corr/chi2_seq:np.float64(0.6067947293631732) - actor/pg_loss:np.float64(-0.00015570060350000858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014229919640911248) - actor/ppo_kl:np.float64(0.00016318686129324078) - actor/pg_clipfrac_lower:np.float64(5.865240382263437e-06) - actor/grad_norm:np.float64(0.6899215131998062) - perf/mfu/actor:np.float64(0.05693643757760258) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.72046184539795) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012832941487431526 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012832941487431526 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.3515625 - response_length/max:1553.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.3515625 - response_length_non_aborted/max:1553.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003503747284412384 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2059585116803646) - timing_s/agent_loop/generate_sequences/max:np.float64(9.883644664660096) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.228372771802242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.883644664660096) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1553 - timing_s/gen:15.838685717433691 - timing_s/reward:0.051979394629597664 - timing_s/old_log_prob:2.7615113761276007 - timing_s/adv:0.1317354552447796 - timing_s/update_actor:9.826176561415195 - timing_s/step:28.72526820190251 - timing_s/stop_profile:0.0001147240400314331 - timing_per_token_ms/gen:0.1919328871989735 - timing_per_token_ms/update_actor:0.06243361265560594 - timing_per_token_ms/adv:0.0008370214329405384 - perf/total_num_tokens:157386 - perf/time_per_step:28.72526820190251 - perf/throughput:684.876111920759 (TaskRunner pid=2099163) Training Progress: 15%|█▌ | 15/100 [14:20<1:00:12, 42.50s/it] (TaskRunner pid=2099163) step:16 - global_seqlen/min:17621 - global_seqlen/max:49777 - global_seqlen/minmax_diff:32156 - global_seqlen/balanced_min:23638 - global_seqlen/balanced_max:29431 - global_seqlen/mean:27074.0 - actor/entropy:0.12440122663974762 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7334084510803223 - training/rollout_probs_diff_mean:0.003286235500127077 - training/rollout_probs_diff_std:0.012201991863548756 - training/rollout_actor_probs_pearson_corr:0.997710645198822 - rollout_corr/rollout_is_mean:0.999930739402771 - rollout_corr/rollout_is_ratio_fraction_high:7.548765097453725e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.793888314859942e-05 - rollout_corr/rollout_is_max:2.695401906967163 - rollout_corr/rollout_is_min:0.02908293716609478 - rollout_corr/rollout_is_std:0.034535329788923264 - rollout_corr/rollout_is_eff_sample_size:0.9988087318703696 - rollout_corr/rollout_is_seq_mean:0.9999657869338989 - rollout_corr/rollout_is_seq_std:0.002763616619631648 - rollout_corr/rollout_is_seq_max:1.0094408988952637 - rollout_corr/rollout_is_seq_min:0.9909818768501282 - rollout_corr/rollout_is_seq_max_deviation:0.009440898895263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2334859780967236) - rollout_corr/training_log_ppl:np.float64(0.20614377667880035) - rollout_corr/kl:np.float64(0.0010040062433311903) - rollout_corr/k3_kl:np.float64(0.0012267602566424785) - rollout_corr/rollout_ppl:np.float64(1.2322044824250042) - rollout_corr/rollout_log_ppl:np.float64(0.20513977024347696) - rollout_corr/log_ppl_diff:np.float64(0.0010040064353233902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025442858877795516) - rollout_corr/log_ppl_diff_max:np.float64(0.014998257160186768) - rollout_corr/log_ppl_diff_min:np.float64(-0.008154168725013733) - rollout_corr/ppl_ratio:np.float64(1.0010099613573402) - rollout_corr/chi2_token:np.float64(0.002799068344756961) - rollout_corr/chi2_seq:np.float64(0.6977890518028289) - actor/pg_loss:np.float64(0.0005094839725643396) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022031578653241013) - actor/ppo_kl:np.float64(-6.424526176118661e-05) - actor/pg_clipfrac_lower:np.float64(1.4852661479380913e-05) - actor/grad_norm:np.float64(0.759627491235733) - perf/mfu/actor:np.float64(0.07361790070915576) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.84224700927734) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0865541398525238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0865541398525238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:517.46875 - response_length/max:8192.0 - response_length/min:85.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:517.46875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.716115891933441e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6790603306144476) - timing_s/agent_loop/generate_sequences/max:np.float64(51.488350415602326) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.263302836421644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.488350415602326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.936794666573405 - timing_s/reward:0.06626211851835251 - timing_s/old_log_prob:2.6734151635318995 - timing_s/adv:0.12808487936854362 - timing_s/update_actor:10.97300206311047 - timing_s/step:71.86347073316574 - timing_s/stop_profile:0.00010935962200164795 - timing_per_token_ms/gen:0.43735124906828166 - timing_per_token_ms/update_actor:0.05066208384017171 - timing_per_token_ms/adv:0.0005913647751003898 - perf/total_num_tokens:216592 - perf/time_per_step:71.86347073316574 - perf/throughput:376.7421712837628 (TaskRunner pid=2099163) Training Progress: 16%|█▌ | 16/100 [15:32<1:11:53, 51.36s/it] (TaskRunner pid=2099163) step:17 - global_seqlen/min:11966 - global_seqlen/max:39775 - global_seqlen/minmax_diff:27809 - global_seqlen/balanced_min:20593 - global_seqlen/balanced_max:26443 - global_seqlen/mean:22197.625 - actor/entropy:0.15671347081661224 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34504491090774536 - training/rollout_probs_diff_mean:0.003963190596550703 - training/rollout_probs_diff_std:0.012594124302268028 - training/rollout_actor_probs_pearson_corr:0.9979807138442993 - rollout_corr/rollout_is_mean:0.9999871253967285 - rollout_corr/rollout_is_ratio_fraction_high:9.640877578931395e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.640877578931395e-06 - rollout_corr/rollout_is_max:2.198077917098999 - rollout_corr/rollout_is_min:0.4044226408004761 - rollout_corr/rollout_is_std:0.03752783685922623 - rollout_corr/rollout_is_eff_sample_size:0.9985936420813145 - rollout_corr/rollout_is_seq_mean:0.9999175071716309 - rollout_corr/rollout_is_seq_std:0.0031496852170675993 - rollout_corr/rollout_is_seq_max:1.0094223022460938 - rollout_corr/rollout_is_seq_min:0.9900813102722168 - rollout_corr/rollout_is_seq_max_deviation:0.009918689727783203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2560751209966838) - rollout_corr/training_log_ppl:np.float64(0.22452797560981708) - rollout_corr/kl:np.float64(0.0013395728782059635) - rollout_corr/k3_kl:np.float64(0.0012000854812015405) - rollout_corr/rollout_ppl:np.float64(1.2543696821667254) - rollout_corr/rollout_log_ppl:np.float64(0.2231884019674908) - rollout_corr/log_ppl_diff:np.float64(0.001339573642326286) - rollout_corr/log_ppl_abs_diff:np.float64(0.002711944489419693) - rollout_corr/log_ppl_diff_max:np.float64(0.01098942756652832) - rollout_corr/log_ppl_diff_min:np.float64(-0.0078660249710083) - rollout_corr/ppl_ratio:np.float64(1.0013459962792695) - rollout_corr/chi2_token:np.float64(0.0021593261044472456) - rollout_corr/chi2_seq:np.float64(0.8300942496862262) - actor/pg_loss:np.float64(0.0006044314941391349) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015271970464709739) - actor/ppo_kl:np.float64(0.00025955110856701324) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.73015196621418) - perf/mfu/actor:np.float64(0.06131922719985038) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.87406206130981) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.28515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06812485307455063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06812485307455063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:405.17578125 - response_length/max:8192.0 - response_length/min:85.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:405.17578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014323554933071136 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9844553098082542) - timing_s/agent_loop/generate_sequences/max:np.float64(49.343952156603336) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.660478450343362) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.343952156603336) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.40832286886871 - timing_s/reward:0.06138620153069496 - timing_s/old_log_prob:2.554391412064433 - timing_s/adv:0.17900926247239113 - timing_s/update_actor:10.665983552113175 - timing_s/step:68.9551175814122 - timing_s/stop_profile:0.0001098737120628357 - timing_per_token_ms/gen:0.5341848432766325 - timing_per_token_ms/update_actor:0.06006263931452788 - timing_per_token_ms/adv:0.001008042878868748 - perf/total_num_tokens:177581 - perf/time_per_step:68.9551175814122 - perf/throughput:321.9141055599284 (TaskRunner pid=2099163) Training Progress: 17%|█▋ | 17/100 [16:41<1:18:23, 56.67s/it] (TaskRunner pid=2099163) step:18 - global_seqlen/min:11273 - global_seqlen/max:36015 - global_seqlen/minmax_diff:24742 - global_seqlen/balanced_min:21872 - global_seqlen/balanced_max:27887 - global_seqlen/mean:22719.125 - actor/entropy:0.1572161316871643 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7516714334487915 - training/rollout_probs_diff_mean:0.003971869591623545 - training/rollout_probs_diff_std:0.01294691488146782 - training/rollout_actor_probs_pearson_corr:0.9978299140930176 - rollout_corr/rollout_is_mean:1.0000897645950317 - rollout_corr/rollout_is_ratio_fraction_high:3.9406924770446494e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9703462385223247e-05 - rollout_corr/rollout_is_max:2.524366617202759 - rollout_corr/rollout_is_min:0.09788884967565536 - rollout_corr/rollout_is_std:0.037467435002326965 - rollout_corr/rollout_is_eff_sample_size:0.998598397072895 - rollout_corr/rollout_is_seq_mean:1.0000863075256348 - rollout_corr/rollout_is_seq_std:0.0033069117926061153 - rollout_corr/rollout_is_seq_max:1.0111510753631592 - rollout_corr/rollout_is_seq_min:0.9837021827697754 - rollout_corr/rollout_is_seq_max_deviation:0.01629781723022461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2480866289697587) - rollout_corr/training_log_ppl:np.float64(0.2177488712950435) - rollout_corr/kl:np.float64(0.0012160134455356797) - rollout_corr/k3_kl:np.float64(0.0012282074120264497) - rollout_corr/rollout_ppl:np.float64(1.2464866428636014) - rollout_corr/rollout_log_ppl:np.float64(0.21653285443244386) - rollout_corr/log_ppl_diff:np.float64(0.0012160168625996448) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027001788184861653) - rollout_corr/log_ppl_diff_max:np.float64(0.01847335696220398) - rollout_corr/log_ppl_diff_min:np.float64(-0.008770108222961426) - rollout_corr/ppl_ratio:np.float64(1.0012231352739036) - rollout_corr/chi2_token:np.float64(0.0024735613260418177) - rollout_corr/chi2_seq:np.float64(1.276715693064034) - actor/pg_loss:np.float64(0.0005491708870977163) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019209535612390027) - actor/ppo_kl:np.float64(0.00030758063345004416) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7345165759325027) - perf/mfu/actor:np.float64(0.06209275037892638) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.8076171875) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02325008623301983 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02325008623301983 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.50390625 - response_length/max:8192.0 - response_length/min:92.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:396.50390625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001363120973110199 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0805652625858784) - timing_s/agent_loop/generate_sequences/max:np.float64(49.01962508261204) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7108128744512214) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.01962508261204) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:54.90000583231449 - timing_s/reward:0.06206467002630234 - timing_s/old_log_prob:2.6254268158227205 - timing_s/adv:0.12246500886976719 - timing_s/update_actor:10.576374450698495 - timing_s/step:68.37408596090972 - timing_s/stop_profile:0.00011956505477428436 - timing_per_token_ms/gen:0.5408601136132652 - timing_per_token_ms/update_actor:0.05819092092399297 - timing_per_token_ms/adv:0.0006737991057631357 - perf/total_num_tokens:181753 - perf/time_per_step:68.37408596090972 - perf/throughput:332.27683676808186 (TaskRunner pid=2099163) Training Progress: 18%|█▊ | 18/100 [17:49<1:22:16, 60.20s/it] (TaskRunner pid=2099163) step:19 - global_seqlen/min:14639 - global_seqlen/max:26849 - global_seqlen/minmax_diff:12210 - global_seqlen/balanced_min:21052 - global_seqlen/balanced_max:21251 - global_seqlen/mean:21094.25 - actor/entropy:0.18874603509902954 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417160868644714 - training/rollout_probs_diff_mean:0.004822679795324802 - training/rollout_probs_diff_std:0.014397743158042431 - training/rollout_actor_probs_pearson_corr:0.9977278113365173 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:1.0811979336722288e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014055574138183147 - rollout_corr/rollout_is_max:2.0913357734680176 - rollout_corr/rollout_is_min:0.040095582604408264 - rollout_corr/rollout_is_std:0.0410170778632164 - rollout_corr/rollout_is_eff_sample_size:0.998320068587282 - rollout_corr/rollout_is_seq_mean:1.0000311136245728 - rollout_corr/rollout_is_seq_std:0.0029985024593770504 - rollout_corr/rollout_is_seq_max:1.010836124420166 - rollout_corr/rollout_is_seq_min:0.987970232963562 - rollout_corr/rollout_is_seq_max_deviation:0.012029767036437988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.255211434327066) - rollout_corr/training_log_ppl:np.float64(0.22383148300286848) - rollout_corr/kl:np.float64(0.0012053162868781442) - rollout_corr/k3_kl:np.float64(0.0011591173564511337) - rollout_corr/rollout_ppl:np.float64(1.253677450120449) - rollout_corr/rollout_log_ppl:np.float64(0.2226261663454352) - rollout_corr/log_ppl_diff:np.float64(0.0012053166574332863) - rollout_corr/log_ppl_abs_diff:np.float64(0.002684240898815915) - rollout_corr/log_ppl_diff_max:np.float64(0.01323661208152771) - rollout_corr/log_ppl_diff_min:np.float64(-0.010094761848449707) - rollout_corr/ppl_ratio:np.float64(1.0012121815234423) - rollout_corr/chi2_token:np.float64(0.0021450589410960674) - rollout_corr/chi2_seq:np.float64(1.123602865729481) - actor/pg_loss:np.float64(0.00028023216873407364) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009424093339021056) - actor/ppo_kl:np.float64(0.0002521510247390779) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5630562603473663) - perf/mfu/actor:np.float64(0.06105586170219322) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.85847091674805) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012870309874415398 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012870309874415398 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:361.2890625 - response_length/max:1883.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.2890625 - response_length_non_aborted/max:1883.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.548874080181122e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0813855342566967) - timing_s/agent_loop/generate_sequences/max:np.float64(11.688677107915282) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5753794663687586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.688677107915282) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:1883 - timing_s/gen:17.670184712857008 - timing_s/reward:0.06079593487083912 - timing_s/old_log_prob:2.4477415289729834 - timing_s/adv:0.12715278938412666 - timing_s/update_actor:9.766590259969234 - timing_s/step:30.157636089250445 - timing_s/stop_profile:0.00011079944670200348 - timing_per_token_ms/gen:0.19104967794201544 - timing_per_token_ms/update_actor:0.05787471858426606 - timing_per_token_ms/adv:0.0007534801508949516 - perf/total_num_tokens:168754 - perf/time_per_step:30.157636089250445 - perf/throughput:699.4662956198663 (TaskRunner pid=2099163) Training Progress: 19%|█▉ | 19/100 [18:20<1:09:06, 51.19s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) To determine the folding stability score of the given protein sequence, one would typically use a computational method or software that calculates the stability based on the amino acid sequence. This score is often derived from energy calculations that consider factors such as hydrogen bonding, hydrophobic interactions, and van der Waals forces. However, without access to specific computational tools or databases that provide such scores for the given sequence, it is not possible to calculate the exact folding stability score. Therefore, the correct answer cannot be determined from the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2099163) step:20 - global_seqlen/min:12298 - global_seqlen/max:30479 - global_seqlen/minmax_diff:18181 - global_seqlen/balanced_min:21810 - global_seqlen/balanced_max:21871 - global_seqlen/mean:21839.375 - actor/entropy:0.18792696297168732 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6172393560409546 - training/rollout_probs_diff_mean:0.004675295669585466 - training/rollout_probs_diff_std:0.013787684962153435 - training/rollout_actor_probs_pearson_corr:0.9978795051574707 - rollout_corr/rollout_is_mean:1.0000436305999756 - rollout_corr/rollout_is_ratio_fraction_high:5.289270120556466e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010578540241112933 - rollout_corr/rollout_is_max:2.6348955631256104 - rollout_corr/rollout_is_min:0.11010421812534332 - rollout_corr/rollout_is_std:0.040804363787174225 - rollout_corr/rollout_is_eff_sample_size:0.998337890270291 - rollout_corr/rollout_is_seq_mean:1.000036597251892 - rollout_corr/rollout_is_seq_std:0.0025871675461530685 - rollout_corr/rollout_is_seq_max:1.009564757347107 - rollout_corr/rollout_is_seq_min:0.9896543622016907 - rollout_corr/rollout_is_seq_max_deviation:0.010345637798309326 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705717207863927) - rollout_corr/training_log_ppl:np.float64(0.2352731467835838) - rollout_corr/kl:np.float64(0.00124775865324378) - rollout_corr/k3_kl:np.float64(0.001086297427093541) - rollout_corr/rollout_ppl:np.float64(1.2689147526398301) - rollout_corr/rollout_log_ppl:np.float64(0.23402538590016775) - rollout_corr/log_ppl_diff:np.float64(0.0012477608834160492) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022727364994352683) - rollout_corr/log_ppl_diff_max:np.float64(0.012642994523048401) - rollout_corr/log_ppl_diff_min:np.float64(-0.007874488830566406) - rollout_corr/ppl_ratio:np.float64(1.0012527105864137) - rollout_corr/chi2_token:np.float64(0.0018538862932473421) - rollout_corr/chi2_seq:np.float64(0.5317726316861808) - actor/pg_loss:np.float64(-4.45549376308918e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001396393277673269) - actor/ppo_kl:np.float64(0.0002731104437279086) - actor/pg_clipfrac_lower:np.float64(7.614522473886609e-06) - actor/grad_norm:np.float64(0.5186081603169441) - perf/mfu/actor:np.float64(0.06376455117784033) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.90739440917969) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3575) - val-aux/sciknoweval/reward/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.18341197265688536) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6885199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03078753156755329) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.4047799999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13095914178704102) - val-aux/sciknoweval/score/mean@16:np.float64(0.3575) - val-aux/sciknoweval/score/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/score/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/score/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.18341197265688536) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6885199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03078753156755329) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.4047799999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13095914178704102) - val-core/sciknoweval/acc/mean@16:np.float64(0.3575) - val-aux/sciknoweval/acc/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.18341197265688536) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6885199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03078753156755329) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.4047799999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13095914178704102) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04906017482220243) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99724) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.017382402655940103) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9658800000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06179985484869913) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98052) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04963623931745396) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.001544538766104626) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9399599999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07259562722286844) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99358) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02825209907076314) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.90684) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07207216802183986) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99862) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010856231271909862) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8730199999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05670226605201907) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.001787064632295094) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:1 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017243020702153444 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017243020702153444 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.26171875 - response_length/max:1936.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.26171875 - response_length_non_aborted/max:1936.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011486373841762543 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1602171901613474) - timing_s/agent_loop/generate_sequences/max:np.float64(12.04150439798832) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.66143894479319) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.04150439798832) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1936 - timing_s/gen:17.918031726032495 - timing_s/reward:0.06160684861242771 - timing_s/old_log_prob:2.4307430386543274 - timing_s/adv:0.11749370023608208 - timing_s/update_actor:9.837341289967299 - timing_s/step:30.453087836503983 - timing_s/testing:84.54373020865023 - timing_s/save_checkpoint:6.91516363248229 - timing_s/stop_profile:0.00016070157289505005 - timing_per_token_ms/gen:0.18954662201851769 - timing_per_token_ms/update_actor:0.05630507563727956 - timing_per_token_ms/adv:0.0006724877671412418 - perf/total_num_tokens:174715 - perf/time_per_step:30.453087836503983 - perf/throughput:717.148130174873 (TaskRunner pid=2099163) Training Progress: 20%|██ | 20/100 [20:22<1:36:35, 72.45s/it] (TaskRunner pid=2099163) step:21 - global_seqlen/min:13059 - global_seqlen/max:30220 - global_seqlen/minmax_diff:17161 - global_seqlen/balanced_min:20197 - global_seqlen/balanced_max:20234 - global_seqlen/mean:20222.625 - actor/entropy:0.2039741724729538 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915150225162506 - training/rollout_probs_diff_mean:0.005005266051739454 - training/rollout_probs_diff_std:0.013932553119957447 - training/rollout_actor_probs_pearson_corr:0.9979515671730042 - rollout_corr/rollout_is_mean:0.9997883439064026 - rollout_corr/rollout_is_ratio_fraction_high:1.1129289305245038e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.3997433185577393 - rollout_corr/rollout_is_min:0.5131387114524841 - rollout_corr/rollout_is_std:0.04085326939821243 - rollout_corr/rollout_is_eff_sample_size:0.9983332565714974 - rollout_corr/rollout_is_seq_mean:0.9999094605445862 - rollout_corr/rollout_is_seq_std:0.0028472074773162603 - rollout_corr/rollout_is_seq_max:1.011277198791504 - rollout_corr/rollout_is_seq_min:0.9916235208511353 - rollout_corr/rollout_is_seq_max_deviation:0.011277198791503906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2804982154630125) - rollout_corr/training_log_ppl:np.float64(0.2435035327434889) - rollout_corr/kl:np.float64(0.0010719042137878887) - rollout_corr/k3_kl:np.float64(0.0011022668704754324) - rollout_corr/rollout_ppl:np.float64(1.2791178887709975) - rollout_corr/rollout_log_ppl:np.float64(0.24243162775383098) - rollout_corr/log_ppl_diff:np.float64(0.0010719049896579236) - rollout_corr/log_ppl_abs_diff:np.float64(0.002581465640105307) - rollout_corr/log_ppl_diff_max:np.float64(0.009952932596206665) - rollout_corr/log_ppl_diff_min:np.float64(-0.010108351707458496) - rollout_corr/ppl_ratio:np.float64(1.0010773693211377) - rollout_corr/chi2_token:np.float64(0.0022938339971005917) - rollout_corr/chi2_seq:np.float64(1.2692464508581907) - actor/pg_loss:np.float64(0.0004962929524481297) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018514558880724508) - actor/ppo_kl:np.float64(6.505578917792576e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7440718114376068) - perf/mfu/actor:np.float64(0.05869452164939222) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.24159622192383) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015914883697405457 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015914883697405457 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:350.98828125 - response_length/max:1700.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.98828125 - response_length_non_aborted/max:1700.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.96875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.297362804412842e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.264117207378149) - timing_s/agent_loop/generate_sequences/max:np.float64(10.401357980445027) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5284495416854043) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.401357980445027) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1700 - timing_s/gen:16.154271429404616 - timing_s/reward:0.05521078407764435 - timing_s/old_log_prob:2.5382862519472837 - timing_s/adv:0.14272014237940311 - timing_s/update_actor:9.808351395651698 - timing_s/step:28.792014434933662 - timing_s/stop_profile:0.00015562400221824646 - timing_per_token_ms/gen:0.17978555451019573 - timing_per_token_ms/update_actor:0.060627338164875345 - timing_per_token_ms/adv:0.0008821811113752735 - perf/total_num_tokens:161781 - perf/time_per_step:28.792014434933662 - perf/throughput:702.3692296938998 (TaskRunner pid=2099163) Training Progress: 21%|██ | 21/100 [20:50<1:18:09, 59.36s/it]