[2026-07-02 23:56:17] Waiting for PID 999999999. [2026-07-02 23:56:17] PID 999999999 finished. Starting Qwen3 generalization queue. [2026-07-02 23:56:17] Model: Qwen/Qwen3-8B [2026-07-02 23:56:17] Datasets: chemistry physics biology material tooluse [2026-07-02 23:56:17] Methods: grpo rlsd_tr [2026-07-02 23:56:17] Starting Qwen/Qwen3-8B chemistry grpo. Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_grpo_Qwen3_8B 2026-07-02 23:56:19,612 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 23:56:29,407 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2224264) TaskRunner hostname: mole-workspace-rw, PID: 2224264 (TaskRunner pid=2224264) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2224264) 'calculate_entropy': False, (TaskRunner pid=2224264) 'calculate_sum_pi_squared': False, (TaskRunner pid=2224264) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2224264) 'async_save': False, (TaskRunner pid=2224264) 'load_contents': ['model', (TaskRunner pid=2224264) 'optimizer', (TaskRunner pid=2224264) 'extra'], (TaskRunner pid=2224264) 'save_contents': ['model', (TaskRunner pid=2224264) 'optimizer', (TaskRunner pid=2224264) 'extra']}, (TaskRunner pid=2224264) 'clip_ratio': 0.2, (TaskRunner pid=2224264) 'clip_ratio_c': 3.0, (TaskRunner pid=2224264) 'clip_ratio_high': 0.28, (TaskRunner pid=2224264) 'clip_ratio_low': 0.2, (TaskRunner pid=2224264) 'data_loader_seed': 42, (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_coeff': 0, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'freeze_vision_tower': False, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'forward_only': False, (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'full_determinism': False, (TaskRunner pid=2224264) 'model_dtype': 'fp32', (TaskRunner pid=2224264) 'offload_policy': False, (TaskRunner pid=2224264) 'optimizer_offload': False, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'seed': 42, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_orig_params': False, (TaskRunner pid=2224264) 'use_torch_compile': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'grad_clip': 1.0, (TaskRunner pid=2224264) 'kl_loss_coef': 0.0, (TaskRunner pid=2224264) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2224264) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2224264) 'loss_scale_factor': None, (TaskRunner pid=2224264) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2224264) 'betas': [0.9, 0.999], (TaskRunner pid=2224264) 'clip_grad': 1.0, (TaskRunner pid=2224264) 'lr': 1e-06, (TaskRunner pid=2224264) 'lr_scheduler_type': 'constant', (TaskRunner pid=2224264) 'lr_warmup_steps': 10, (TaskRunner pid=2224264) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2224264) 'min_lr_ratio': 0.0, (TaskRunner pid=2224264) 'num_cycles': 0.5, (TaskRunner pid=2224264) 'optimizer': 'AdamW', (TaskRunner pid=2224264) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2224264) 'override_optimizer_config': None, (TaskRunner pid=2224264) 'total_training_steps': -1, (TaskRunner pid=2224264) 'warmup_style': None, (TaskRunner pid=2224264) 'weight_decay': 0.01}, (TaskRunner pid=2224264) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2224264) 'clip_cov_lb': 1.0, (TaskRunner pid=2224264) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2224264) 'clip_cov_ub': 5.0, (TaskRunner pid=2224264) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2224264) 'loss_mode': 'vanilla', (TaskRunner pid=2224264) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2224264) 'ppo_epochs': 1, (TaskRunner pid=2224264) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2224264) 'ppo_micro_batch_size': None, (TaskRunner pid=2224264) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2224264) 'ppo_mini_batch_size': 8, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'rollout_n': 8, (TaskRunner pid=2224264) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2224264) 'mode': 'disabled', (TaskRunner pid=2224264) 'record_file': None, (TaskRunner pid=2224264) 'replay_file': None}, (TaskRunner pid=2224264) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2224264) 'alpha': 0.5, (TaskRunner pid=2224264) 'distillation_add_tail': True, (TaskRunner pid=2224264) 'distillation_topk': 100, (TaskRunner pid=2224264) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2224264) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2224264) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'loss_weight': 0.0, (TaskRunner pid=2224264) 'mask': 'all'}, (TaskRunner pid=2224264) 'feedback_template': '\n' (TaskRunner pid=2224264) 'The ' (TaskRunner pid=2224264) 'following ' (TaskRunner pid=2224264) 'is ' (TaskRunner pid=2224264) 'feedback ' (TaskRunner pid=2224264) 'from ' (TaskRunner pid=2224264) 'your ' (TaskRunner pid=2224264) 'unsuccessful ' (TaskRunner pid=2224264) 'earlier ' (TaskRunner pid=2224264) 'attempt:\n' (TaskRunner pid=2224264) '\n' (TaskRunner pid=2224264) '{feedback_raw}', (TaskRunner pid=2224264) 'full_logit_distillation': True, (TaskRunner pid=2224264) 'include_environment_feedback': True, (TaskRunner pid=2224264) 'is_clip': 2, (TaskRunner pid=2224264) 'max_reprompt_len': 22528, (TaskRunner pid=2224264) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2224264) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2224264) '\n' (TaskRunner pid=2224264) 'Correctly ' (TaskRunner pid=2224264) 'solve ' (TaskRunner pid=2224264) 'the ' (TaskRunner pid=2224264) 'original ' (TaskRunner pid=2224264) 'question.', (TaskRunner pid=2224264) 'reprompt_truncation': 'right', (TaskRunner pid=2224264) 'solution_template': '\n' (TaskRunner pid=2224264) 'Correct ' (TaskRunner pid=2224264) 'solution:\n' (TaskRunner pid=2224264) '\n' (TaskRunner pid=2224264) '{successful_previous_attempt}', (TaskRunner pid=2224264) 'srpo_dynamic_weighting': False, (TaskRunner pid=2224264) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2224264) 'success_reward_threshold': 0.5, (TaskRunner pid=2224264) 'teacher_regularization': 'ema', (TaskRunner pid=2224264) 'teacher_update_rate': 0.0, (TaskRunner pid=2224264) 'token_reweight_decay_steps': None, (TaskRunner pid=2224264) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2224264) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2224264) 'shuffle': False, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2224264) 'tau_neg': 1.05, (TaskRunner pid=2224264) 'tau_pos': 1.0, (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_dynamic_bsz': False, (TaskRunner pid=2224264) 'use_fused_kernels': False, (TaskRunner pid=2224264) 'use_kl_loss': False, (TaskRunner pid=2224264) 'use_prefix_grouper': False, (TaskRunner pid=2224264) 'use_remove_padding': True, (TaskRunner pid=2224264) 'use_torch_compile': True}, (TaskRunner pid=2224264) 'hybrid_engine': True, (TaskRunner pid=2224264) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2224264) 'custom_chat_template': None, (TaskRunner pid=2224264) 'enable_activation_offload': False, (TaskRunner pid=2224264) 'enable_gradient_checkpointing': True, (TaskRunner pid=2224264) 'exclude_modules': None, (TaskRunner pid=2224264) 'external_lib': None, (TaskRunner pid=2224264) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2224264) 'hf_config_path': None, (TaskRunner pid=2224264) 'lora_adapter_path': None, (TaskRunner pid=2224264) 'lora_alpha': 16, (TaskRunner pid=2224264) 'lora_rank': 0, (TaskRunner pid=2224264) 'override_config': {}, (TaskRunner pid=2224264) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'target_modules': 'all-linear', (TaskRunner pid=2224264) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2224264) 'num_shards': 4}, (TaskRunner pid=2224264) 'tokenizer_path': None, (TaskRunner pid=2224264) 'trust_remote_code': True, (TaskRunner pid=2224264) 'use_fused_kernels': False, (TaskRunner pid=2224264) 'use_liger': False, (TaskRunner pid=2224264) 'use_remove_padding': True, (TaskRunner pid=2224264) 'use_shm': False}, (TaskRunner pid=2224264) 'nccl_timeout': 600, (TaskRunner pid=2224264) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'forward_only': True, (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'full_determinism': False, (TaskRunner pid=2224264) 'model_dtype': 'fp32', (TaskRunner pid=2224264) 'offload_policy': False, (TaskRunner pid=2224264) 'optimizer_offload': False, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'seed': 42, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_orig_params': False, (TaskRunner pid=2224264) 'use_torch_compile': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2224264) 'log_prob_micro_batch_size': None, (TaskRunner pid=2224264) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2224264) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'rollout_n': 8, (TaskRunner pid=2224264) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2224264) 'mode': 'disabled', (TaskRunner pid=2224264) 'record_file': None, (TaskRunner pid=2224264) 'replay_file': None}, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_torch_compile': True}, (TaskRunner pid=2224264) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2224264) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2224264) 'agent_loop_config_path': None, (TaskRunner pid=2224264) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2224264) 'name': None, (TaskRunner pid=2224264) 'path': None}, (TaskRunner pid=2224264) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2224264) 'num_workers': 8}, (TaskRunner pid=2224264) 'calculate_log_probs': True, (TaskRunner pid=2224264) 'cudagraph_capture_sizes': None, (TaskRunner pid=2224264) 'data_parallel_size': 1, (TaskRunner pid=2224264) 'disable_log_stats': True, (TaskRunner pid=2224264) 'do_sample': True, (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'enable_chunked_prefill': True, (TaskRunner pid=2224264) 'enable_prefix_caching': True, (TaskRunner pid=2224264) 'enable_rollout_routing_replay': False, (TaskRunner pid=2224264) 'enforce_eager': False, (TaskRunner pid=2224264) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2224264) 'expert_parallel_size': 1, (TaskRunner pid=2224264) 'free_cache_engine': True, (TaskRunner pid=2224264) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2224264) 'ignore_eos': False, (TaskRunner pid=2224264) 'layered_summon': False, (TaskRunner pid=2224264) 'load_format': 'dummy', (TaskRunner pid=2224264) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2224264) 'log_prob_micro_batch_size': None, (TaskRunner pid=2224264) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2224264) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2224264) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2224264) 'max_model_len': 10240, (TaskRunner pid=2224264) 'max_num_batched_tokens': 10240, (TaskRunner pid=2224264) 'max_num_seqs': 1024, (TaskRunner pid=2224264) 'mode': 'async', (TaskRunner pid=2224264) 'multi_stage_wake_up': False, (TaskRunner pid=2224264) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'format': 'hermes', (TaskRunner pid=2224264) 'interaction_config_path': None, (TaskRunner pid=2224264) 'max_assistant_turns': None, (TaskRunner pid=2224264) 'max_parallel_calls': 1, (TaskRunner pid=2224264) 'max_tool_response_length': 256, (TaskRunner pid=2224264) 'max_user_turns': None, (TaskRunner pid=2224264) 'num_repeat_rollouts': None, (TaskRunner pid=2224264) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2224264) 'tool_config_path': None, (TaskRunner pid=2224264) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2224264) 'use_inference_chat_template': False}, (TaskRunner pid=2224264) 'n': 8, (TaskRunner pid=2224264) 'name': 'vllm', (TaskRunner pid=2224264) 'over_sample_rate': 0, (TaskRunner pid=2224264) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2224264) 'port': 9090, (TaskRunner pid=2224264) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2224264) 'prompt_length': 2048, (TaskRunner pid=2224264) 'quantization': None, (TaskRunner pid=2224264) 'quantization_config_file': None, (TaskRunner pid=2224264) 'response_length': 8192, (TaskRunner pid=2224264) 'scheduling_policy': 'fcfs', (TaskRunner pid=2224264) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2224264) 'skip_rollout': False, (TaskRunner pid=2224264) 'skip_tokenizer_init': True, (TaskRunner pid=2224264) 'temperature': 1.0, (TaskRunner pid=2224264) 'tensor_model_parallel_size': 2, (TaskRunner pid=2224264) 'top_k': -1, (TaskRunner pid=2224264) 'top_p': 1.0, (TaskRunner pid=2224264) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2224264) 'backend': None, (TaskRunner pid=2224264) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2224264) 'token2text': False}, (TaskRunner pid=2224264) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2224264) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2224264) 'do_sample': True, (TaskRunner pid=2224264) 'n': 16, (TaskRunner pid=2224264) 'temperature': 0.6, (TaskRunner pid=2224264) 'top_k': -1, (TaskRunner pid=2224264) 'top_p': 0.95}}}, (TaskRunner pid=2224264) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2224264) 'adv_estimator': 'grpo', (TaskRunner pid=2224264) 'gamma': 1.0, (TaskRunner pid=2224264) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2224264) 'horizon': 10000, (TaskRunner pid=2224264) 'kl_coef': 0.001, (TaskRunner pid=2224264) 'target_kl': 0.1, (TaskRunner pid=2224264) 'type': 'fixed'}, (TaskRunner pid=2224264) 'kl_penalty': 'kl', (TaskRunner pid=2224264) 'lam': 1.0, (TaskRunner pid=2224264) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2224264) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2224264) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2224264) 'loss_type': 'ppo_clip', (TaskRunner pid=2224264) 'rollout_is': 'token', (TaskRunner pid=2224264) 'rollout_is_batch_normalize': False, (TaskRunner pid=2224264) 'rollout_is_threshold': 2.0, (TaskRunner pid=2224264) 'rollout_rs': None, (TaskRunner pid=2224264) 'rollout_rs_threshold': None}, (TaskRunner pid=2224264) 'use_kl_in_reward': False, (TaskRunner pid=2224264) 'use_pf_ppo': False}, (TaskRunner pid=2224264) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2224264) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2224264) 'async_save': False, (TaskRunner pid=2224264) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2224264) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2224264) 'cliprange_value': 0.5, (TaskRunner pid=2224264) 'data_loader_seed': 42, (TaskRunner pid=2224264) 'enable': None, (TaskRunner pid=2224264) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2224264) 'forward_micro_batch_size': None, (TaskRunner pid=2224264) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2224264) 'grad_clip': 1.0, (TaskRunner pid=2224264) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2224264) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2224264) 'enable_activation_offload': False, (TaskRunner pid=2224264) 'enable_gradient_checkpointing': True, (TaskRunner pid=2224264) 'external_lib': None, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'forward_only': False, (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'full_determinism': False, (TaskRunner pid=2224264) 'model_dtype': 'fp32', (TaskRunner pid=2224264) 'offload_policy': False, (TaskRunner pid=2224264) 'optimizer_offload': False, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'seed': 42, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_orig_params': False, (TaskRunner pid=2224264) 'use_torch_compile': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'lora_alpha': 16, (TaskRunner pid=2224264) 'lora_rank': 0, (TaskRunner pid=2224264) 'override_config': {}, (TaskRunner pid=2224264) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'target_modules': 'all-linear', (TaskRunner pid=2224264) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2224264) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'trust_remote_code': True, (TaskRunner pid=2224264) 'use_remove_padding': False, (TaskRunner pid=2224264) 'use_shm': False}, (TaskRunner pid=2224264) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2224264) 'betas': [0.9, 0.999], (TaskRunner pid=2224264) 'clip_grad': 1.0, (TaskRunner pid=2224264) 'lr': 1e-05, (TaskRunner pid=2224264) 'lr_scheduler_type': 'constant', (TaskRunner pid=2224264) 'lr_warmup_steps': -1, (TaskRunner pid=2224264) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2224264) 'min_lr_ratio': 0.0, (TaskRunner pid=2224264) 'num_cycles': 0.5, (TaskRunner pid=2224264) 'optimizer': 'AdamW', (TaskRunner pid=2224264) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2224264) 'override_optimizer_config': None, (TaskRunner pid=2224264) 'total_training_steps': -1, (TaskRunner pid=2224264) 'warmup_style': None, (TaskRunner pid=2224264) 'weight_decay': 0.01}, (TaskRunner pid=2224264) 'ppo_epochs': 1, (TaskRunner pid=2224264) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2224264) 'ppo_micro_batch_size': None, (TaskRunner pid=2224264) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2224264) 'ppo_mini_batch_size': 8, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'rollout_n': 8, (TaskRunner pid=2224264) 'shuffle': False, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_dynamic_bsz': False}, (TaskRunner pid=2224264) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2224264) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2224264) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2224264) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2224264) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2224264) 'dataloader_num_workers': 8, (TaskRunner pid=2224264) 'filter_overlong_prompts': True, (TaskRunner pid=2224264) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2224264) 'image_key': 'images', (TaskRunner pid=2224264) 'image_patch_size': 14, (TaskRunner pid=2224264) 'max_prompt_length': 2048, (TaskRunner pid=2224264) 'max_response_length': 8192, (TaskRunner pid=2224264) 'prompt_key': 'prompt', (TaskRunner pid=2224264) 'return_full_prompt': False, (TaskRunner pid=2224264) 'return_multi_modal_inputs': True, (TaskRunner pid=2224264) 'return_raw_chat': True, (TaskRunner pid=2224264) 'return_raw_input_ids': False, (TaskRunner pid=2224264) 'reward_fn_key': 'data_source', (TaskRunner pid=2224264) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2224264) 'seed': None, (TaskRunner pid=2224264) 'shuffle': True, (TaskRunner pid=2224264) 'tokenizer': None, (TaskRunner pid=2224264) 'tool_config_path': None, (TaskRunner pid=2224264) 'train_batch_size': 32, (TaskRunner pid=2224264) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2224264) 'train_max_samples': 3200, (TaskRunner pid=2224264) 'truncation': 'error', (TaskRunner pid=2224264) 'trust_remote_code': True, (TaskRunner pid=2224264) 'use_shm': False, (TaskRunner pid=2224264) 'val_batch_size': None, (TaskRunner pid=2224264) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2224264) 'val_max_samples': -1, (TaskRunner pid=2224264) 'validation_shuffle': False, (TaskRunner pid=2224264) 'video_key': 'videos'}, (TaskRunner pid=2224264) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2224264) 'cuda-memory-usage': 'true', (TaskRunner pid=2224264) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2224264) 'capture-range-end': None, (TaskRunner pid=2224264) 'cuda-graph-trace': 'graph', (TaskRunner pid=2224264) 'cuda-memory-usage': 'true', (TaskRunner pid=2224264) 'kill': 'none', (TaskRunner pid=2224264) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2224264) 'torch_memory': {'context': 'all', (TaskRunner pid=2224264) 'kw_args': {}, (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'stacks': 'all', (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2224264) 'profile_continuous_steps': False, (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'steps': None, (TaskRunner pid=2224264) 'tool': None}, (TaskRunner pid=2224264) 'max_model_len': 10240, (TaskRunner pid=2224264) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_8B', (TaskRunner pid=2224264) 'include_dashboard': False, (TaskRunner pid=2224264) 'num_cpus': None, (TaskRunner pid=2224264) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2224264) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2224264) 'USER': 'root'}}}, (TaskRunner pid=2224264) 'timeline_json_file': None}, (TaskRunner pid=2224264) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2224264) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2224264) 'name': 'custom_reward_manager', (TaskRunner pid=2224264) 'path': None}, (TaskRunner pid=2224264) 'name': 'naive', (TaskRunner pid=2224264) 'source': 'register'}, (TaskRunner pid=2224264) 'reward_model': {'enable': False, (TaskRunner pid=2224264) 'enable_resource_pool': False, (TaskRunner pid=2224264) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2224264) 'launch_reward_fn_async': False, (TaskRunner pid=2224264) 'max_length': None, (TaskRunner pid=2224264) 'micro_batch_size': None, (TaskRunner pid=2224264) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2224264) 'model': {'external_lib': None, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'override_config': {}, (TaskRunner pid=2224264) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2224264) 'trust_remote_code': False, (TaskRunner pid=2224264) 'use_fused_kernels': False, (TaskRunner pid=2224264) 'use_remove_padding': False, (TaskRunner pid=2224264) 'use_shm': False}, (TaskRunner pid=2224264) 'n_gpus_per_node': 8, (TaskRunner pid=2224264) 'nnodes': 0, (TaskRunner pid=2224264) 'num_workers': 1, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'reward_loop_class_name': None, (TaskRunner pid=2224264) 'reward_loop_module_path': None, (TaskRunner pid=2224264) 'reward_loop_source': 'register', (TaskRunner pid=2224264) 'reward_manager': 'naive', (TaskRunner pid=2224264) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2224264) 'cudagraph_capture_sizes': None, (TaskRunner pid=2224264) 'data_parallel_size': 1, (TaskRunner pid=2224264) 'disable_log_stats': True, (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'enable_chunked_prefill': True, (TaskRunner pid=2224264) 'enable_prefix_caching': True, (TaskRunner pid=2224264) 'enforce_eager': True, (TaskRunner pid=2224264) 'engine_kwargs': {}, (TaskRunner pid=2224264) 'expert_parallel_size': 1, (TaskRunner pid=2224264) 'free_cache_engine': True, (TaskRunner pid=2224264) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2224264) 'limit_images': None, (TaskRunner pid=2224264) 'load_format': 'auto', (TaskRunner pid=2224264) 'max_model_len': None, (TaskRunner pid=2224264) 'max_num_batched_tokens': 8192, (TaskRunner pid=2224264) 'max_num_seqs': 1024, (TaskRunner pid=2224264) 'name': '???', (TaskRunner pid=2224264) 'prompt_length': 2048, (TaskRunner pid=2224264) 'response_length': 2048, (TaskRunner pid=2224264) 'skip_tokenizer_init': False, (TaskRunner pid=2224264) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2224264) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2224264) 'memory_limit_mb': 1024, (TaskRunner pid=2224264) 'url': None}, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_dynamic_bsz': False, (TaskRunner pid=2224264) 'use_reward_loop': False}, (TaskRunner pid=2224264) 'trainer': {'balance_batch': True, (TaskRunner pid=2224264) 'critic_warmup': 0, (TaskRunner pid=2224264) 'default_hdfs_dir': None, (TaskRunner pid=2224264) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2224264) 'del_local_ckpt_after_load': False, (TaskRunner pid=2224264) 'device': 'cuda', (TaskRunner pid=2224264) 'esi_redundant_time': 0, (TaskRunner pid=2224264) 'experiment_name': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2224264) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2224264) 'log_val_generations': 0, (TaskRunner pid=2224264) 'logger': ['console', 'wandb'], (TaskRunner pid=2224264) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2224264) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2224264) 'n_gpus_per_node': 8, (TaskRunner pid=2224264) 'nnodes': 1, (TaskRunner pid=2224264) 'project_name': 'SDPO-root', (TaskRunner pid=2224264) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2224264) 'resume_from_path': None, (TaskRunner pid=2224264) 'resume_mode': 'auto', (TaskRunner pid=2224264) 'rollout_data_dir': None, (TaskRunner pid=2224264) 'save_freq': 10, (TaskRunner pid=2224264) 'test_freq': 10, (TaskRunner pid=2224264) 'total_epochs': 30, (TaskRunner pid=2224264) 'total_training_steps': 100, (TaskRunner pid=2224264) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2224264) 'val_before_train': False, (TaskRunner pid=2224264) 'val_only': False, (TaskRunner pid=2224264) 'validation_data_dir': None}, (TaskRunner pid=2224264) 'transfer_queue': {'enable': False}, (TaskRunner pid=2224264) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2224264) 'dir': '/users/root/SDPO', (TaskRunner pid=2224264) 'log_dir': '/users/root/output', (TaskRunner pid=2224264) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2224264) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2224264) use_critic=need_critic(config), (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2224264) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2224264) Using dataset class: RLHFDataset (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2224264) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2224264) dataset len: 1890 (TaskRunner pid=2224264) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2224264) WARNING:2026-07-02 23:56:38,560:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2224264) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2224264) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method wake_up to class .WorkerDict'> (WorkerDict pid=2224678) [W702 23:56:48.468298935 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:38433 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2224673) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2224673) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2224680) [W702 23:56:48.948596036 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:38433 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2224673) Model config after override: Qwen3Config { (WorkerDict pid=2224673) "architectures": [ (WorkerDict pid=2224673) "Qwen3ForCausalLM" (WorkerDict pid=2224673) ], (WorkerDict pid=2224673) "attention_bias": false, (WorkerDict pid=2224673) "attention_dropout": 0.0, (WorkerDict pid=2224673) "dtype": "bfloat16", (WorkerDict pid=2224673) "eos_token_id": 151645, (WorkerDict pid=2224673) "head_dim": 128, (WorkerDict pid=2224673) "hidden_act": "silu", (WorkerDict pid=2224673) "hidden_size": 4096, (WorkerDict pid=2224673) "initializer_range": 0.02, (WorkerDict pid=2224673) "intermediate_size": 12288, (WorkerDict pid=2224673) "layer_types": [ (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention" (WorkerDict pid=2224673) ], (WorkerDict pid=2224673) "max_position_embeddings": 40960, (WorkerDict pid=2224673) "max_window_layers": 36, (WorkerDict pid=2224673) "model_type": "qwen3", (WorkerDict pid=2224673) "num_attention_heads": 32, (WorkerDict pid=2224673) "num_hidden_layers": 36, (WorkerDict pid=2224673) "num_key_value_heads": 8, (WorkerDict pid=2224673) "pad_token_id": 151643, (WorkerDict pid=2224673) "rms_norm_eps": 1e-06, (WorkerDict pid=2224673) "rope_scaling": null, (WorkerDict pid=2224673) "rope_theta": 1000000, (WorkerDict pid=2224673) "sliding_window": null, (WorkerDict pid=2224673) "tie_word_embeddings": false, (WorkerDict pid=2224673) "transformers_version": "4.57.1", (WorkerDict pid=2224673) "use_cache": true, (WorkerDict pid=2224673) "use_sliding_window": false, (WorkerDict pid=2224673) "vocab_size": 151936 (WorkerDict pid=2224673) } (WorkerDict pid=2224673) (WorkerDict pid=2224673) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2224673) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2224673) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2224673) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2224673) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2224674) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2224674) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2224680) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.93s/it] [repeated 15x across cluster] (WorkerDict pid=2224680) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.62s/it] [repeated 7x across cluster] (WorkerDict pid=2224673) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2224673) Actor use_remove_padding=True (WorkerDict pid=2224673) Actor use_fused_kernels=False (WorkerDict pid=2224673) Actor use_prefix_grouper=False (WorkerDict pid=2224680) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2224680) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2224674) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2224674) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2224674) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2224673) warnings.warn( (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2224673) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2224673) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2224680) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2224680) warnings.warn( [repeated 7x across cluster] (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2224264) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (vLLMHttpServer pid=2225569) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2225569) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2225570) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2225570) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2225569) ['serve', (vLLMHttpServer pid=2225569) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2225569) '--dtype', (vLLMHttpServer pid=2225569) 'bfloat16', (vLLMHttpServer pid=2225569) '--load_format', (vLLMHttpServer pid=2225569) 'dummy', (vLLMHttpServer pid=2225569) '--trust_remote_code', (vLLMHttpServer pid=2225569) '--max_model_len', (vLLMHttpServer pid=2225569) '40960', (vLLMHttpServer pid=2225569) '--max_num_seqs', (vLLMHttpServer pid=2225569) '1024', (vLLMHttpServer pid=2225569) '--enable_chunked_prefill', (vLLMHttpServer pid=2225569) '--max_num_batched_tokens', (vLLMHttpServer pid=2225569) '10240', (vLLMHttpServer pid=2225569) '--enable_prefix_caching', (vLLMHttpServer pid=2225569) '--enable_sleep_mode', (vLLMHttpServer pid=2225569) '--logprobs_mode', (vLLMHttpServer pid=2225569) 'processed_logprobs', (vLLMHttpServer pid=2225569) '--disable_custom_all_reduce', (vLLMHttpServer pid=2225569) '--gpu_memory_utilization', (vLLMHttpServer pid=2225569) '0.8', (vLLMHttpServer pid=2225569) '--disable_log_stats', (vLLMHttpServer pid=2225569) '--tensor_parallel_size', (vLLMHttpServer pid=2225569) '2', (vLLMHttpServer pid=2225569) '--seed', (vLLMHttpServer pid=2225569) '0', (vLLMHttpServer pid=2225569) '--override_generation_config', (vLLMHttpServer pid=2225569) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2225569) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2225569) '--hf_overrides', (vLLMHttpServer pid=2225569) '{}', (vLLMHttpServer pid=2225569) '--scheduling_policy', (vLLMHttpServer pid=2225569) 'fcfs'] (WorkerDict pid=2224680) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2225569) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2225569) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2225569) WARNING 07-02 23:57:38 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2224673) WARNING 07-02 23:57:46 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2225571) WARNING 07-02 23:57:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2224675) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2224677) 2026-07-02 23:58:01,108 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (vLLMHttpServer pid=2225571) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2225571) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2225571) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2225571) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2224678) 2026-07-02 23:58:01,122 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2224673) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and identify what functional groups or atoms are present that must be introduced in the synthesis. (TaskRunner pid=2224264) (TaskRunner pid=2224264) The product has the following key features: (TaskRunner pid=2224264) - A nitrogen-containing ring (indole-like structure) (TaskRunner pid=2224264) - A chlorine atom attached to a benzene ring (TaskRunner pid=2224264) - A trifluoromethyl group (C(F)(F)F) attached to a benzene ring (TaskRunner pid=2224264) - A substituent group that includes a nitrogen atom bonded to a carbon chain (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are given four options (A, B, C, D) and need to identify which one is the only correct reactant that can be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option A:** (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) This option includes the same ring structure as the product, but it has an extra oxygen and a double bond. It seems to be a more complex derivative and may not be a direct reactant for the synthesis of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option B:** (TaskRunner pid=2224264) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) This option has a chlorine atom attached to a carbon that is also bonded to a nitrogen. It includes a ring structure and a chlorine atom, but it does not match the exact structure of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option C:** (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) This option includes a trifluoromethyl group (C(F)(F)F), a chlorine atom, and a ring structure. However, it has an oxygen atom attached to a carbon that is also bonded to a ring. This structure is more complex and seems to be a derivative of the product, not a direct reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option D:** (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) This option contains a ring structure, a trifluoromethyl group (C(F)(F)F), and a carbon chain with a double bond. It matches the necessary components of the product, including the trifluoromethyl group and the ring structure. It is the only option that contains the trifluoromethyl group and a carbon chain that could be used to build the product's structure. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Thus, the only correct reactant that matches the necessary components of the product is **Option D**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) D (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 0.0 (TaskRunner pid=2224264) [acc] 0.0 (TaskRunner pid=2224264) [pred] D (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2224264) step:10 - global_seqlen/min:20418 - global_seqlen/max:28068 - global_seqlen/minmax_diff:7650 - global_seqlen/balanced_min:23873 - global_seqlen/balanced_max:23970 - global_seqlen/mean:23892.875 - actor/entropy:0.5462746024131775 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977370262146 - training/rollout_probs_diff_mean:0.004709707107394934 - training/rollout_probs_diff_std:0.009677772410213947 - training/rollout_actor_probs_pearson_corr:0.9994706511497498 - rollout_corr/rollout_is_mean:0.9998794198036194 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.18444459582679e-06 - rollout_corr/rollout_is_max:1.622496247291565 - rollout_corr/rollout_is_min:0.4166650176048279 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987795959879308 - rollout_corr/rollout_is_seq_mean:0.9998565912246704 - rollout_corr/rollout_is_seq_std:0.0018025203607976437 - rollout_corr/rollout_is_seq_max:1.0046859979629517 - rollout_corr/rollout_is_seq_min:0.9944931268692017 - rollout_corr/rollout_is_seq_max_deviation:0.00550687313079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7906524906866252) - rollout_corr/training_log_ppl:np.float64(0.5681954241008498) - rollout_corr/kl:np.float64(0.000816460652924178) - rollout_corr/k3_kl:np.float64(0.0006859729254529157) - rollout_corr/rollout_ppl:np.float64(1.789125895127654) - rollout_corr/rollout_log_ppl:np.float64(0.5673789661959745) - rollout_corr/log_ppl_diff:np.float64(0.0008164579048752785) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015062049496918917) - rollout_corr/log_ppl_diff_max:np.float64(0.007841110229492188) - rollout_corr/log_ppl_diff_min:np.float64(-0.004483282566070557) - rollout_corr/ppl_ratio:np.float64(1.0008185552433133) - rollout_corr/chi2_token:np.float64(0.0011271920520812273) - rollout_corr/chi2_seq:np.float64(0.35812679631635547) - actor/pg_loss:np.float64(-9.637698531150818e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003609476652854937) - actor/ppo_kl:np.float64(1.5247296355269668e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38365066051483154) - perf/mfu/actor:np.float64(0.09690410487686144) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.56629180908203) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42410714285714285) - val-aux/sciknoweval/reward/std@16:np.float64(0.3064350931481341) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5519095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2843083193630651) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.2967190476190476) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23547573506551264) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.42388571428571425) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3061951415792409) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6715952380952382) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.229658540263386) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.21003333333333332) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1505707856063924) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43492857142857144) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.25799645877611216) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.765652380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1562366777957225) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1566142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09130497085868691) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.44051904761904753) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20808730912319104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8246428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0841254257162172) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.12242857142857143) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05650288713800214) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.44238571428571427) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1575830385066246) - val-aux/sciknoweval/score/mean@16:np.float64(0.42410714285714285) - val-aux/sciknoweval/score/std@16:np.float64(0.3064350931481341) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5519095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2843083193630651) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.2967190476190476) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.23547573506551264) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.42388571428571425) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3061951415792409) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6715952380952382) - val-aux/sciknoweval/score/best@4/std:np.float64(0.229658540263386) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.21003333333333332) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1505707856063924) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43492857142857144) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.25799645877611216) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.765652380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1562366777957225) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1566142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09130497085868691) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.44051904761904753) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20808730912319104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8246428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0841254257162172) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.12242857142857143) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05650288713800214) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.44238571428571427) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1575830385066246) - val-core/sciknoweval/acc/mean@16:np.float64(0.42410714285714285) - val-aux/sciknoweval/acc/std@16:np.float64(0.3064350931481341) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5519095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2843083193630651) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.2967190476190476) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.23547573506551264) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.42388571428571425) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3061951415792409) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6715952380952382) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.229658540263386) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.21003333333333332) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1505707856063924) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43492857142857144) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.25799645877611216) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.765652380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1562366777957225) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1566142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09130497085868691) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.44051904761904753) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20808730912319104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8246428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0841254257162172) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.12242857142857143) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05650288713800214) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.44238571428571427) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1575830385066246) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9955357142857143) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.017290104224140252) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.999647619047619) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.004935643403728794) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9915619047619048) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.02303424670029897) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9958761904761906) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.016639024114376826) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9841142857142856) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02968846984091712) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9983333333333334) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.010724673876175908) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9717047619047618) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.03493004126518059) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995761904761905) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005142725136555151) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9547523809523809) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03440111225850822) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999714285714284) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0007119567591960759) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003161241766065359 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003161241766065359 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.27734375 - response_length/max:1573.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.27734375 - response_length_non_aborted/max:1573.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015650317072868347 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0671779178082943) - timing_s/agent_loop/generate_sequences/max:np.float64(11.94185172766447) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.342340253017028) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.94185172766447) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1573 - timing_s/gen:18.750355437397957 - timing_s/reward:0.06060847267508507 - timing_s/old_log_prob:3.169859381392598 - timing_s/adv:0.15408215299248695 - timing_s/update_actor:12.863298328593373 - timing_s/step:35.09146223217249 - timing_s/testing:148.6573334839195 - timing_s/save_checkpoint:12.811893673613667 - timing_s/stop_profile:0.00013393908739089966 - timing_per_token_ms/adv:0.0008061093160224908 - timing_per_token_ms/gen:0.1534612461422453 - timing_per_token_ms/update_actor:0.0672967272073441 - perf/total_num_tokens:191143 - perf/time_per_step:35.09146223217249 - perf/throughput:680.8743061750952 (TaskRunner pid=2224264) Training Progress: 10%|█ | 10/100 [09:38<2:10:46, 87.18s/it] (TaskRunner pid=2224264) step:11 - global_seqlen/min:20294 - global_seqlen/max:26725 - global_seqlen/minmax_diff:6431 - global_seqlen/balanced_min:24058 - global_seqlen/balanced_max:24076 - global_seqlen/mean:24070.25 - actor/entropy:0.5913231372833252 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23963481187820435 - training/rollout_probs_diff_mean:0.004752492066472769 - training/rollout_probs_diff_std:0.009357922710478306 - training/rollout_actor_probs_pearson_corr:0.9995139241218567 - rollout_corr/rollout_is_mean:1.0000724792480469 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7242271496797912e-05 - rollout_corr/rollout_is_max:1.639595627784729 - rollout_corr/rollout_is_min:0.41323140263557434 - rollout_corr/rollout_is_std:0.03530850633978844 - rollout_corr/rollout_is_eff_sample_size:0.9987550993777402 - rollout_corr/rollout_is_seq_mean:1.0001287460327148 - rollout_corr/rollout_is_seq_std:0.001979757333174348 - rollout_corr/rollout_is_seq_max:1.0079063177108765 - rollout_corr/rollout_is_seq_min:0.9954867959022522 - rollout_corr/rollout_is_seq_max_deviation:0.007906317710876465 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8813741342164576) - rollout_corr/training_log_ppl:np.float64(0.6172284478088841) - rollout_corr/kl:np.float64(0.0006792962190935725) - rollout_corr/k3_kl:np.float64(0.0007297945928712579) - rollout_corr/rollout_ppl:np.float64(1.8801453905180097) - rollout_corr/rollout_log_ppl:np.float64(0.6165491492720321) - rollout_corr/log_ppl_diff:np.float64(0.0006792985368520021) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017337198369204998) - rollout_corr/log_ppl_diff_max:np.float64(0.00989621877670288) - rollout_corr/log_ppl_diff_min:np.float64(-0.006999552249908447) - rollout_corr/ppl_ratio:np.float64(1.000681943492964) - rollout_corr/chi2_token:np.float64(0.0015983630437403917) - rollout_corr/chi2_seq:np.float64(0.8742716582491994) - actor/pg_loss:np.float64(-0.00019931478891521692) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003874804629049322) - actor/ppo_kl:np.float64(0.000144808165869037) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4361106902360916) - perf/mfu/actor:np.float64(0.09751332153344734) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(105.7747859954834) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005713657476007938 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005713657476007938 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.1015625 - response_length/max:1181.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.1015625 - response_length_non_aborted/max:1181.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.448345422744751e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.323524136096239) - timing_s/agent_loop/generate_sequences/max:np.float64(8.916998555883765) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.45590021499811) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.916998555883765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:319 - timing_s/agent_loop/slowest/response_length:1181 - timing_s/gen:15.152160251513124 - timing_s/reward:0.057105498388409615 - timing_s/old_log_prob:3.2711402103304863 - timing_s/adv:0.15576941706240177 - timing_s/update_actor:13.000678863376379 - timing_s/step:31.738592451438308 - timing_s/stop_profile:0.00011883489787578583 - timing_per_token_ms/adv:0.0008089312380552849 - timing_per_token_ms/gen:0.13062882779724058 - timing_per_token_ms/update_actor:0.0675142492463538 - perf/total_num_tokens:192562 - perf/time_per_step:31.738592451438308 - perf/throughput:758.3905945680714 (TaskRunner pid=2224264) Training Progress: 11%|█ | 11/100 [10:09<1:44:10, 70.23s/it] (TaskRunner pid=2224264) step:12 - global_seqlen/min:22279 - global_seqlen/max:28767 - global_seqlen/minmax_diff:6488 - global_seqlen/balanced_min:25313 - global_seqlen/balanced_max:25329 - global_seqlen/mean:25325.625 - actor/entropy:0.5468142628669739 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30783188343048096 - training/rollout_probs_diff_mean:0.004490185528993607 - training/rollout_probs_diff_std:0.009273474104702473 - training/rollout_actor_probs_pearson_corr:0.9995062351226807 - rollout_corr/rollout_is_mean:1.0000845193862915 - rollout_corr/rollout_is_ratio_fraction_high:7.435773568431614e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.075246810913086 - rollout_corr/rollout_is_min:0.5100579261779785 - rollout_corr/rollout_is_std:0.03405739739537239 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:1.0000801086425781 - rollout_corr/rollout_is_seq_std:0.0016866717487573624 - rollout_corr/rollout_is_seq_max:1.0045456886291504 - rollout_corr/rollout_is_seq_min:0.9936270713806152 - rollout_corr/rollout_is_seq_max_deviation:0.006372928619384766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7922574547119439) - rollout_corr/training_log_ppl:np.float64(0.5712134827044792) - rollout_corr/kl:np.float64(0.0005522373108419742) - rollout_corr/k3_kl:np.float64(0.0006720679998579726) - rollout_corr/rollout_ppl:np.float64(1.7912592180073261) - rollout_corr/rollout_log_ppl:np.float64(0.5706612397334538) - rollout_corr/log_ppl_diff:np.float64(0.0005522429710254073) - rollout_corr/log_ppl_abs_diff:np.float64(0.001352141029201448) - rollout_corr/log_ppl_diff_max:np.float64(0.007192671298980713) - rollout_corr/log_ppl_diff_min:np.float64(-0.003842592239379883) - rollout_corr/ppl_ratio:np.float64(1.0005537504330277) - rollout_corr/chi2_token:np.float64(0.0016048443503677845) - rollout_corr/chi2_seq:np.float64(0.7803806539159268) - actor/pg_loss:np.float64(-5.2671064622700214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002636122353578685) - actor/ppo_kl:np.float64(2.2634695866941e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3025449775159359) - perf/mfu/actor:np.float64(0.09925995879832478) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.3304214477539) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003494813572615385 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003494813572615385 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:525.33203125 - response_length/max:1340.0 - response_length/min:219.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:525.33203125 - response_length_non_aborted/max:1340.0 - response_length_non_aborted/min:219.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001250188797712326 - timing_s/agent_loop/generate_sequences/min:np.float64(2.742388065904379) - timing_s/agent_loop/generate_sequences/max:np.float64(11.409071335569024) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.0465768424546695) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.409071335569024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1340 - timing_s/gen:18.710343282669783 - timing_s/reward:0.06669484823942184 - timing_s/old_log_prob:3.1704280357807875 - timing_s/adv:0.14182846806943417 - timing_s/update_actor:13.454730957746506 - timing_s/step:35.63072715699673 - timing_s/stop_profile:0.0001154579222202301 - timing_per_token_ms/adv:0.0007000245209616454 - timing_per_token_ms/gen:0.1391258748757838 - timing_per_token_ms/update_actor:0.06640868170946672 - perf/total_num_tokens:202605 - perf/time_per_step:35.63072715699673 - perf/throughput:710.7804701377491 (TaskRunner pid=2224264) Training Progress: 12%|█▏ | 12/100 [10:45<1:27:35, 59.72s/it] (TaskRunner pid=2224264) step:13 - global_seqlen/min:22139 - global_seqlen/max:26070 - global_seqlen/minmax_diff:3931 - global_seqlen/balanced_min:23703 - global_seqlen/balanced_max:23955 - global_seqlen/mean:23768.875 - actor/entropy:0.6082448959350586 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8117851614952087 - training/rollout_probs_diff_mean:0.004782064817845821 - training/rollout_probs_diff_std:0.009643850848078728 - training/rollout_actor_probs_pearson_corr:0.9995208978652954 - rollout_corr/rollout_is_mean:0.9998765587806702 - rollout_corr/rollout_is_ratio_fraction_high:8.387642992602196e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.677528598520439e-05 - rollout_corr/rollout_is_max:2.2560038566589355 - rollout_corr/rollout_is_min:0.1834239512681961 - rollout_corr/rollout_is_std:0.03520030155777931 - rollout_corr/rollout_is_eff_sample_size:0.9987622341887205 - rollout_corr/rollout_is_seq_mean:0.9998448491096497 - rollout_corr/rollout_is_seq_std:0.0017469775630161166 - rollout_corr/rollout_is_seq_max:1.0048439502716064 - rollout_corr/rollout_is_seq_min:0.9949353933334351 - rollout_corr/rollout_is_seq_max_deviation:0.005064606666564941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8870359733700752) - rollout_corr/training_log_ppl:np.float64(0.6238076707813889) - rollout_corr/kl:np.float64(0.0008689244228179405) - rollout_corr/k3_kl:np.float64(0.0006925447481762603) - rollout_corr/rollout_ppl:np.float64(1.8853362454101443) - rollout_corr/rollout_log_ppl:np.float64(0.6229387456551194) - rollout_corr/log_ppl_diff:np.float64(0.0008689251262694597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016076937317848206) - rollout_corr/log_ppl_diff_max:np.float64(0.007483184337615967) - rollout_corr/log_ppl_diff_min:np.float64(-0.0051155686378479) - rollout_corr/ppl_ratio:np.float64(1.00087097636424) - rollout_corr/chi2_token:np.float64(0.001052779145538807) - rollout_corr/chi2_seq:np.float64(0.7514310674741864) - actor/pg_loss:np.float64(7.94560182839632e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002786977966025006) - actor/ppo_kl:np.float64(6.474458803040761e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.397162064909935) - perf/mfu/actor:np.float64(0.09482056304606831) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.62731170654297) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025341168511658907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025341168511658907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:465.71484375 - response_length/max:1517.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.71484375 - response_length_non_aborted/max:1517.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010670162737369537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1883809324353933) - timing_s/agent_loop/generate_sequences/max:np.float64(11.769639683887362) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.302525144354149) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.769639683887362) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:1517 - timing_s/gen:18.91310457699001 - timing_s/reward:0.0668586827814579 - timing_s/old_log_prob:3.0675366707146168 - timing_s/adv:0.13432571291923523 - timing_s/update_actor:13.279257319867611 - timing_s/step:35.54763768427074 - timing_s/stop_profile:0.00011998414993286133 - timing_per_token_ms/adv:0.0007064160215788254 - timing_per_token_ms/gen:0.15863637533856728 - timing_per_token_ms/update_actor:0.06983532729182393 - perf/total_num_tokens:190151 - perf/time_per_step:35.54763768427074 - perf/throughput:668.6485107987174 (TaskRunner pid=2224264) Training Progress: 13%|█▎ | 13/100 [11:21<1:15:59, 52.41s/it] (TaskRunner pid=2224264) step:14 - global_seqlen/min:21108 - global_seqlen/max:28388 - global_seqlen/minmax_diff:7280 - global_seqlen/balanced_min:24288 - global_seqlen/balanced_max:24367 - global_seqlen/mean:24304.0 - actor/entropy:0.5927061438560486 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932337522506714 - training/rollout_probs_diff_mean:0.004692592658102512 - training/rollout_probs_diff_std:0.00925843883305788 - training/rollout_actor_probs_pearson_corr:0.9995315670967102 - rollout_corr/rollout_is_mean:0.9999940991401672 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.629692554473877 - rollout_corr/rollout_is_min:0.5129373669624329 - rollout_corr/rollout_is_std:0.03493344411253929 - rollout_corr/rollout_is_eff_sample_size:0.998781141930825 - rollout_corr/rollout_is_seq_mean:0.9999703168869019 - rollout_corr/rollout_is_seq_std:0.0018831780180335045 - rollout_corr/rollout_is_seq_max:1.007323145866394 - rollout_corr/rollout_is_seq_min:0.9952291250228882 - rollout_corr/rollout_is_seq_max_deviation:0.007323145866394043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8519614487886429) - rollout_corr/training_log_ppl:np.float64(0.6055547567666508) - rollout_corr/kl:np.float64(0.0006922458704323731) - rollout_corr/k3_kl:np.float64(0.0006847217604217803) - rollout_corr/rollout_ppl:np.float64(1.850690824445337) - rollout_corr/rollout_log_ppl:np.float64(0.604862505744677) - rollout_corr/log_ppl_diff:np.float64(0.0006922510219737887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016473830910399556) - rollout_corr/log_ppl_diff_max:np.float64(0.005706071853637695) - rollout_corr/log_ppl_diff_min:np.float64(-0.006285727024078369) - rollout_corr/ppl_ratio:np.float64(1.00069451588206) - rollout_corr/chi2_token:np.float64(0.0013829697854816914) - rollout_corr/chi2_seq:np.float64(1.670509943505749) - actor/pg_loss:np.float64(4.7211069613695145e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002791377244193427) - actor/ppo_kl:np.float64(2.9361409389139226e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39511965960264206) - perf/mfu/actor:np.float64(0.09905766411058545) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.6309814453125) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001839753007516265 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.001839753007516265 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:492.0625 - response_length/max:1506.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:492.0625 - response_length_non_aborted/max:1506.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010376237332820892 - timing_s/agent_loop/generate_sequences/min:np.float64(2.323818039149046) - timing_s/agent_loop/generate_sequences/max:np.float64(11.8414571788162) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.66649279979174) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.8414571788162) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1506 - timing_s/gen:18.678927469998598 - timing_s/reward:0.09569268859922886 - timing_s/old_log_prob:3.05848809145391 - timing_s/adv:0.16165275499224663 - timing_s/update_actor:12.951634736731648 - timing_s/step:35.03177675232291 - timing_s/stop_profile:0.00011714547872543335 - timing_per_token_ms/adv:0.0008314102359295107 - timing_per_token_ms/gen:0.1482831153149895 - timing_per_token_ms/update_actor:0.06661267042838447 - perf/total_num_tokens:194432 - perf/time_per_step:35.03177675232291 - perf/throughput:693.7701211055027 (TaskRunner pid=2224264) Training Progress: 14%|█▍ | 14/100 [11:56<1:07:37, 47.17s/it] (TaskRunner pid=2224264) step:15 - global_seqlen/min:22401 - global_seqlen/max:28174 - global_seqlen/minmax_diff:5773 - global_seqlen/balanced_min:25422 - global_seqlen/balanced_max:25432 - global_seqlen/mean:25428.5 - actor/entropy:0.644464373588562 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21453610062599182 - training/rollout_probs_diff_mean:0.004607322160154581 - training/rollout_probs_diff_std:0.008982905186712742 - training/rollout_actor_probs_pearson_corr:0.9995591640472412 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.693491170357447e-06 - rollout_corr/rollout_is_max:1.5525251626968384 - rollout_corr/rollout_is_min:0.4232032895088196 - rollout_corr/rollout_is_std:0.0347040556371212 - rollout_corr/rollout_is_eff_sample_size:0.9987969583907628 - rollout_corr/rollout_is_seq_mean:1.000001072883606 - rollout_corr/rollout_is_seq_std:0.001659586327150464 - rollout_corr/rollout_is_seq_max:1.0077872276306152 - rollout_corr/rollout_is_seq_min:0.994013249874115 - rollout_corr/rollout_is_seq_max_deviation:0.007787227630615234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.001432800665498) - rollout_corr/training_log_ppl:np.float64(0.6742313308641315) - rollout_corr/kl:np.float64(0.0006003224210004987) - rollout_corr/k3_kl:np.float64(0.0006980515448731239) - rollout_corr/rollout_ppl:np.float64(2.0002612890675664) - rollout_corr/rollout_log_ppl:np.float64(0.673631009995006) - rollout_corr/log_ppl_diff:np.float64(0.0006003208691254258) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014080120017752051) - rollout_corr/log_ppl_diff_max:np.float64(0.00482177734375) - rollout_corr/log_ppl_diff_min:np.float64(-0.007164597511291504) - rollout_corr/ppl_ratio:np.float64(1.0006020197179168) - rollout_corr/chi2_token:np.float64(0.001624844502657652) - rollout_corr/chi2_seq:np.float64(1.5461109012831002) - actor/pg_loss:np.float64(0.00015092291869223118) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017774101524992147) - actor/ppo_kl:np.float64(-3.1671785084341764e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3353484757244587) - perf/mfu/actor:np.float64(0.10308922077230062) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.59129333496094) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006222110823728144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006222110823728144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:507.734375 - response_length/max:1102.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:507.734375 - response_length_non_aborted/max:1102.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000116739422082901 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3703849464654922) - timing_s/agent_loop/generate_sequences/max:np.float64(9.983016459271312) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.813961479689169) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.983016459271312) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1093 - timing_s/gen:17.158777246251702 - timing_s/reward:0.06794460862874985 - timing_s/old_log_prob:3.08405601978302 - timing_s/adv:0.12691123597323895 - timing_s/update_actor:12.908528231084347 - timing_s/step:33.43038131669164 - timing_s/stop_profile:2.7742236852645874e-05 - timing_per_token_ms/adv:0.0006238631652144196 - timing_per_token_ms/gen:0.13201090357171644 - timing_per_token_ms/update_actor:0.06345502207702158 - perf/total_num_tokens:203428 - perf/time_per_step:33.43038131669164 - perf/throughput:760.6404413731191 (TaskRunner pid=2224264) Training Progress: 15%|█▌ | 15/100 [12:29<1:00:58, 43.04s/it] (TaskRunner pid=2224264) step:16 - global_seqlen/min:21410 - global_seqlen/max:32573 - global_seqlen/minmax_diff:11163 - global_seqlen/balanced_min:26094 - global_seqlen/balanced_max:26108 - global_seqlen/mean:26103.625 - actor/entropy:0.6026089787483215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1841939091682434 - training/rollout_probs_diff_mean:0.004535277374088764 - training/rollout_probs_diff_std:0.008988428860902786 - training/rollout_actor_probs_pearson_corr:0.9995559453964233 - rollout_corr/rollout_is_mean:1.000097632408142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6370495557785034 - rollout_corr/rollout_is_min:0.5366882681846619 - rollout_corr/rollout_is_std:0.034516338258981705 - rollout_corr/rollout_is_eff_sample_size:0.9988102779034601 - rollout_corr/rollout_is_seq_mean:1.0000861883163452 - rollout_corr/rollout_is_seq_std:0.00168060592841357 - rollout_corr/rollout_is_seq_max:1.0056310892105103 - rollout_corr/rollout_is_seq_min:0.9956023097038269 - rollout_corr/rollout_is_seq_max_deviation:0.005631089210510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8813290460966527) - rollout_corr/training_log_ppl:np.float64(0.6181044758995995) - rollout_corr/kl:np.float64(0.0006577574873949388) - rollout_corr/k3_kl:np.float64(0.0006780388332572329) - rollout_corr/rollout_ppl:np.float64(1.880119965877384) - rollout_corr/rollout_log_ppl:np.float64(0.617446719086729) - rollout_corr/log_ppl_diff:np.float64(0.0006577568128705025) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014887393917888403) - rollout_corr/log_ppl_diff_max:np.float64(0.009718984365463257) - rollout_corr/log_ppl_diff_min:np.float64(-0.005425512790679932) - rollout_corr/ppl_ratio:np.float64(1.0006597633473575) - rollout_corr/chi2_token:np.float64(0.0014098691754043102) - rollout_corr/chi2_seq:np.float64(2.303532388759777) - actor/pg_loss:np.float64(-0.00024759862571954727) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005523854347302404) - actor/ppo_kl:np.float64(0.00011259556639942048) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4002230614423752) - perf/mfu/actor:np.float64(0.10440002990848575) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.71824264526367) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004673639312386513 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004673639312386513 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.01953125 - response_length/max:1208.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.01953125 - response_length_non_aborted/max:1208.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.0385330319404602e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0918461363762617) - timing_s/agent_loop/generate_sequences/max:np.float64(10.83283576183021) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.100973397937196) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.83283576183021) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:1163 - timing_s/gen:17.82087113149464 - timing_s/reward:0.06884737312793732 - timing_s/old_log_prob:3.0925582088530064 - timing_s/adv:0.11866064369678497 - timing_s/update_actor:13.111870618537068 - timing_s/step:34.29907225072384 - timing_s/stop_profile:0.00011087954044342041 - timing_per_token_ms/adv:0.0005682191826651709 - timing_per_token_ms/gen:0.1301125917679308 - timing_per_token_ms/update_actor:0.0627875947236115 - perf/total_num_tokens:208829 - perf/time_per_step:34.29907225072384 - perf/throughput:761.059214931072 (TaskRunner pid=2224264) Training Progress: 16%|█▌ | 16/100 [13:04<56:35, 40.42s/it] (TaskRunner pid=2224264) step:17 - global_seqlen/min:25658 - global_seqlen/max:31580 - global_seqlen/minmax_diff:5922 - global_seqlen/balanced_min:28427 - global_seqlen/balanced_max:28619 - global_seqlen/mean:28480.125 - actor/entropy:0.6307497024536133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24447917938232422 - training/rollout_probs_diff_mean:0.004340663086622953 - training/rollout_probs_diff_std:0.008799507282674313 - training/rollout_actor_probs_pearson_corr:0.9995622634887695 - rollout_corr/rollout_is_mean:1.0000449419021606 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.015722076175734e-06 - rollout_corr/rollout_is_max:1.5821566581726074 - rollout_corr/rollout_is_min:0.4949151575565338 - rollout_corr/rollout_is_std:0.03397679701447487 - rollout_corr/rollout_is_eff_sample_size:0.998847027329727 - rollout_corr/rollout_is_seq_mean:1.0000169277191162 - rollout_corr/rollout_is_seq_std:0.0016468465328216553 - rollout_corr/rollout_is_seq_max:1.006962537765503 - rollout_corr/rollout_is_seq_min:0.9948194622993469 - rollout_corr/rollout_is_seq_max_deviation:0.00696253776550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0382777974009514) - rollout_corr/training_log_ppl:np.float64(0.6885525062098168) - rollout_corr/kl:np.float64(0.0007926652352825414) - rollout_corr/k3_kl:np.float64(0.000665645356832556) - rollout_corr/rollout_ppl:np.float64(2.0366465840488672) - rollout_corr/rollout_log_ppl:np.float64(0.687759839813225) - rollout_corr/log_ppl_diff:np.float64(0.0007926663965918124) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014466078137047589) - rollout_corr/log_ppl_diff_max:np.float64(0.0072629451751708984) - rollout_corr/log_ppl_diff_min:np.float64(-0.004837632179260254) - rollout_corr/ppl_ratio:np.float64(1.0007944458629936) - rollout_corr/chi2_token:np.float64(0.0010911454446613789) - rollout_corr/chi2_seq:np.float64(0.8875894017983228) - actor/pg_loss:np.float64(4.744785837829113e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019340229709996493) - actor/ppo_kl:np.float64(0.00017584012368843105) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3084394074976444) - perf/mfu/actor:np.float64(0.1127631964826109) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.56375503540039) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005952840205281973 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005952840205281973 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:556.78515625 - response_length/max:1943.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:556.78515625 - response_length_non_aborted/max:1943.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012286566197872162 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8024302516132593) - timing_s/agent_loop/generate_sequences/max:np.float64(15.304179856553674) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.37867253353761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.304179856553674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1943 - timing_s/gen:21.94722494482994 - timing_s/reward:0.07538571581244469 - timing_s/old_log_prob:3.1670491863042116 - timing_s/adv:0.15483485721051693 - timing_s/update_actor:13.154093807563186 - timing_s/step:38.58247912488878 - timing_s/stop_profile:0.00011283159255981445 - timing_per_token_ms/adv:0.0006795741644853952 - timing_per_token_ms/gen:0.15397563400962516 - timing_per_token_ms/update_actor:0.057733655521013276 - perf/total_num_tokens:227841 - perf/time_per_step:38.58247912488878 - perf/throughput:738.1621307384586 (TaskRunner pid=2224264) Training Progress: 17%|█▋ | 17/100 [13:42<55:10, 39.88s/it] (TaskRunner pid=2224264) step:18 - global_seqlen/min:21346 - global_seqlen/max:28688 - global_seqlen/minmax_diff:7342 - global_seqlen/balanced_min:24408 - global_seqlen/balanced_max:24420 - global_seqlen/mean:24413.25 - actor/entropy:0.6875190734863281 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4119926691055298 - training/rollout_probs_diff_mean:0.0047507998533546925 - training/rollout_probs_diff_std:0.009137631393969059 - training/rollout_actor_probs_pearson_corr:0.9995434284210205 - rollout_corr/rollout_is_mean:0.9999079704284668 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3933751435833983e-05 - rollout_corr/rollout_is_max:1.649957537651062 - rollout_corr/rollout_is_min:0.2244005799293518 - rollout_corr/rollout_is_std:0.03564453125 - rollout_corr/rollout_is_eff_sample_size:0.9987308417828792 - rollout_corr/rollout_is_seq_mean:0.9998347759246826 - rollout_corr/rollout_is_seq_std:0.0020229120273143053 - rollout_corr/rollout_is_seq_max:1.005630373954773 - rollout_corr/rollout_is_seq_min:0.9910240769386292 - rollout_corr/rollout_is_seq_max_deviation:0.00897592306137085 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0345947919413447) - rollout_corr/training_log_ppl:np.float64(0.6928353062248789) - rollout_corr/kl:np.float64(0.0008856774063996653) - rollout_corr/k3_kl:np.float64(0.0006838213192850162) - rollout_corr/rollout_ppl:np.float64(2.0328480429016054) - rollout_corr/rollout_log_ppl:np.float64(0.6919496294576675) - rollout_corr/log_ppl_diff:np.float64(0.0008856767672114074) - rollout_corr/log_ppl_abs_diff:np.float64(0.001704056456219405) - rollout_corr/log_ppl_diff_max:np.float64(0.00990024209022522) - rollout_corr/log_ppl_diff_min:np.float64(-0.004864394664764404) - rollout_corr/ppl_ratio:np.float64(1.0008882319089025) - rollout_corr/chi2_token:np.float64(0.0009777550585567951) - rollout_corr/chi2_seq:np.float64(0.8223780991975218) - actor/pg_loss:np.float64(0.00021487544290721416) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000207112968837464) - actor/ppo_kl:np.float64(5.649118667616637e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38337405771017075) - perf/mfu/actor:np.float64(0.09893896312961671) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.62813949584961) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014918704982846975 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014918704982846975 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:489.6328125 - response_length/max:1187.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:489.6328125 - response_length_non_aborted/max:1187.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001486111432313919 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7897587660700083) - timing_s/agent_loop/generate_sequences/max:np.float64(10.173580570146441) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.444523438432952) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.173580570146441) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:1187 - timing_s/gen:17.149094123393297 - timing_s/reward:0.10750813968479633 - timing_s/old_log_prob:3.131292313337326 - timing_s/adv:0.1582884006202221 - timing_s/update_actor:12.998078908771276 - timing_s/step:33.6287423055619 - timing_s/stop_profile:0.00010932423174381256 - timing_per_token_ms/adv:0.0008104635834035928 - timing_per_token_ms/gen:0.13681405169206276 - timing_per_token_ms/update_actor:0.06655237887607793 - perf/total_num_tokens:195306 - perf/time_per_step:33.6287423055619 - perf/throughput:725.9638132813031 (TaskRunner pid=2224264) Training Progress: 18%|█▊ | 18/100 [14:16<51:58, 38.03s/it] (TaskRunner pid=2224264) step:19 - global_seqlen/min:21987 - global_seqlen/max:30303 - global_seqlen/minmax_diff:8316 - global_seqlen/balanced_min:26121 - global_seqlen/balanced_max:26132 - global_seqlen/mean:26128.5 - actor/entropy:0.6984733939170837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413516640663147 - training/rollout_probs_diff_mean:0.004623448941856623 - training/rollout_probs_diff_std:0.008824565447866917 - training/rollout_actor_probs_pearson_corr:0.9995649456977844 - rollout_corr/rollout_is_mean:0.9998148083686829 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.593936970806681e-06 - rollout_corr/rollout_is_max:1.9665265083312988 - rollout_corr/rollout_is_min:0.4158303737640381 - rollout_corr/rollout_is_std:0.035356584936380386 - rollout_corr/rollout_is_eff_sample_size:0.9987510563634333 - rollout_corr/rollout_is_seq_mean:0.9998022317886353 - rollout_corr/rollout_is_seq_std:0.001719683175906539 - rollout_corr/rollout_is_seq_max:1.005003571510315 - rollout_corr/rollout_is_seq_min:0.9942381978034973 - rollout_corr/rollout_is_seq_max_deviation:0.0057618021965026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1011477317661047) - rollout_corr/training_log_ppl:np.float64(0.7186182939331047) - rollout_corr/kl:np.float64(0.000854842338526618) - rollout_corr/k3_kl:np.float64(0.0006763917490957283) - rollout_corr/rollout_ppl:np.float64(2.0992823550477624) - rollout_corr/rollout_log_ppl:np.float64(0.7177634529653005) - rollout_corr/log_ppl_diff:np.float64(0.0008548409678041935) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015009507769718766) - rollout_corr/log_ppl_diff_max:np.float64(0.0058748722076416016) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035433173179626465) - rollout_corr/ppl_ratio:np.float64(1.000856626080349) - rollout_corr/chi2_token:np.float64(0.0010158277582377195) - rollout_corr/chi2_seq:np.float64(0.722900487948209) - actor/pg_loss:np.float64(0.00023403007071465254) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021148656901459617) - actor/ppo_kl:np.float64(1.1429972420096135e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.30054471269249916) - perf/mfu/actor:np.float64(0.10550384895416316) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.6054801940918) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012871723156422377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012871723156422377 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:514.390625 - response_length/max:1276.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:514.390625 - response_length_non_aborted/max:1276.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001385621726512909 - timing_s/agent_loop/generate_sequences/min:np.float64(1.989379409700632) - timing_s/agent_loop/generate_sequences/max:np.float64(10.74332383275032) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9351765791507205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.74332383275032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:1276 - timing_s/gen:17.415069840848446 - timing_s/reward:0.06816395185887814 - timing_s/old_log_prob:3.1230949088931084 - timing_s/adv:0.12103297747671604 - timing_s/update_actor:13.004424184560776 - timing_s/step:33.81597386114299 - timing_s/stop_profile:3.315135836601257e-05 - timing_per_token_ms/adv:0.0005790275823177567 - timing_per_token_ms/gen:0.13224894323417002 - timing_per_token_ms/update_actor:0.062213790423104924 - perf/total_num_tokens:209028 - perf/time_per_step:33.81597386114299 - perf/throughput:772.667382204939 (TaskRunner pid=2224264) Training Progress: 19%|█▉ | 19/100 [14:50<49:38, 36.77s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) To determine which molecule is the only correct reactant used in the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure and functional groups of the product and match it with the structure of the reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 1: Analyze the product (TaskRunner pid=2224264) The product is: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This can be interpreted as: (TaskRunner pid=2224264) - A nitrogen atom (N) is attached to a ring. (TaskRunner pid=2224264) - The ring is a pyrrole-like structure (N in the ring). (TaskRunner pid=2224264) - The ring is substituted with a group: `Cc2ccc(Cl)cc2C(F)(F)F`, which is a benzene ring with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2224264) - The structure is a pyrrole derivative with a substituent on the ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 2: Analyze the options (TaskRunner pid=2224264) Let’s examine the options one by one to see which one matches the structure of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option A: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) - A carbonyl group (O=C), which is not in the product. (TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product. (TaskRunner pid=2224264) - The part `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` matches the pyrrole-like structure of the product, but the presence of two carbonyl groups makes this a different compound. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option B: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This contains: (TaskRunner pid=2224264) - A carbonyl group (O=C), which is not in the product. (TaskRunner pid=2224264) - A chlorine atom (Cl) attached to the ring. (TaskRunner pid=2224264) - The rest of the structure does not match the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option C: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This contains: (TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product. (TaskRunner pid=2224264) - A hydroxyl group (O), which is not in the product. (TaskRunner pid=2224264) - The part `c2ccc(Cl)c(C(F)(F)F)c2` is similar to the product, but the overall structure is not matching the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option D: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This contains: (TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product. (TaskRunner pid=2224264) - A fluorinated group (C(F)(F)F), which is present in the product. (TaskRunner pid=2224264) - The structure is not matching the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 3: Conclusion (TaskRunner pid=2224264) None of the options exactly match the structure of the product. However, **Option A** is the only one that contains the core structure of the product: `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1`. The other groups in Option A (carbonyl groups) are not part of the product, but the rest of the structure is the closest match. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Thus, **Option A** is the best candidate as the correct reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2224264) step:20 - global_seqlen/min:22848 - global_seqlen/max:30898 - global_seqlen/minmax_diff:8050 - global_seqlen/balanced_min:26847 - global_seqlen/balanced_max:26876 - global_seqlen/mean:26867.0 - actor/entropy:0.6798826456069946 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967540204524994 - training/rollout_probs_diff_mean:0.004492460284382105 - training/rollout_probs_diff_std:0.008851698599755764 - training/rollout_actor_probs_pearson_corr:0.9995761513710022 - rollout_corr/rollout_is_mean:0.9998908638954163 - rollout_corr/rollout_is_ratio_fraction_high:1.4051259313418996e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.54838228225708 - rollout_corr/rollout_is_min:0.5467735528945923 - rollout_corr/rollout_is_std:0.034748680889606476 - rollout_corr/rollout_is_eff_sample_size:0.9987937474899505 - rollout_corr/rollout_is_seq_mean:0.9998890161514282 - rollout_corr/rollout_is_seq_std:0.0017418391071259975 - rollout_corr/rollout_is_seq_max:1.0073792934417725 - rollout_corr/rollout_is_seq_min:0.9951199293136597 - rollout_corr/rollout_is_seq_max_deviation:0.007379293441772461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.033839966636151) - rollout_corr/training_log_ppl:np.float64(0.6936833750805818) - rollout_corr/kl:np.float64(0.0007494871477833343) - rollout_corr/k3_kl:np.float64(0.0006487431917321373) - rollout_corr/rollout_ppl:np.float64(2.0322980885393918) - rollout_corr/rollout_log_ppl:np.float64(0.6929338824702427) - rollout_corr/log_ppl_diff:np.float64(0.0007494926103390753) - rollout_corr/log_ppl_abs_diff:np.float64(0.001494129013735801) - rollout_corr/log_ppl_diff_max:np.float64(0.005956470966339111) - rollout_corr/log_ppl_diff_min:np.float64(-0.006281614303588867) - rollout_corr/ppl_ratio:np.float64(1.000751372659579) - rollout_corr/chi2_token:np.float64(0.0011060182005167007) - rollout_corr/chi2_seq:np.float64(0.7747026174329221) - actor/pg_loss:np.float64(-0.00010517542250454426) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014331985744320264) - actor/ppo_kl:np.float64(1.3654966693366077e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31467055529356003) - perf/mfu/actor:np.float64(0.10809592356334449) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.8171272277832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.48392857142857143) - val-aux/sciknoweval/reward/std@16:np.float64(0.3171121217029239) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6190095238095239) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2728610239160176) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3498904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2656159472576298) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.48431904761904765) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3171144172517289) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.729252380952381) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.20041976709665926) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.24425714285714284) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1861647179153658) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5091) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2646122540406198) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8068619047619048) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.12410804929329204) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.17493809523809525) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11508084398834663) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5237238095238095) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20562351058186512) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8522714285714285) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06491422684529762) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.13282380952380954) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06387633477899576) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5326761904761904) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.15389547364520514) - val-aux/sciknoweval/score/mean@16:np.float64(0.48392857142857143) - val-aux/sciknoweval/score/std@16:np.float64(0.3171121217029239) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6190095238095239) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2728610239160176) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3498904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2656159472576298) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.48431904761904765) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3171144172517289) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.729252380952381) - val-aux/sciknoweval/score/best@4/std:np.float64(0.20041976709665926) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.24425714285714284) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1861647179153658) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5091) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2646122540406198) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8068619047619048) - val-aux/sciknoweval/score/best@8/std:np.float64(0.12410804929329204) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.17493809523809525) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11508084398834663) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5237238095238095) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20562351058186512) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8522714285714285) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06491422684529762) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.13282380952380954) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06387633477899576) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5326761904761904) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.15389547364520514) - val-core/sciknoweval/acc/mean@16:np.float64(0.48392857142857143) - val-aux/sciknoweval/acc/std@16:np.float64(0.3171121217029239) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6190095238095239) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2728610239160176) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3498904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2656159472576298) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.48431904761904765) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3171144172517289) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.729252380952381) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.20041976709665926) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.24425714285714284) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1861647179153658) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5091) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2646122540406198) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8068619047619048) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.12410804929329204) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.17493809523809525) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11508084398834663) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5237238095238095) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20562351058186512) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8522714285714285) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06491422684529762) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.13282380952380954) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06387633477899576) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5326761904761904) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.15389547364520514) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9931547619047619) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.023312712167358005) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9993809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006485809156183406) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9871333333333333) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.030787750802297818) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9933857142857144) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.02299957497117341) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9763380952380952) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03858134164350246) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9975285714285714) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.014190325800172833) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9601904761904761) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.043352249329006456) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995333333333334) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005554643730532354) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9392047619047619) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04061806930568887) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999666666666667) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0009652924861912784) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017669458175078034 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017669458175078034 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:556.0 - response_length/max:1546.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:556.0 - response_length_non_aborted/max:1546.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.37429004907608e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.663187799975276) - timing_s/agent_loop/generate_sequences/max:np.float64(12.704742550849915) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.302203726881999) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.704742550849915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:336 - timing_s/agent_loop/slowest/response_length:1546 - timing_s/gen:19.35746957361698 - timing_s/reward:0.06838529743254185 - timing_s/old_log_prob:3.1675357930362225 - timing_s/adv:0.14193121530115604 - timing_s/update_actor:13.063340313732624 - timing_s/step:35.8843690212816 - timing_s/testing:160.607242802158 - timing_s/save_checkpoint:12.75176297314465 - timing_s/stop_profile:0.0001437067985534668 - timing_per_token_ms/adv:0.0006603417542950275 - timing_per_token_ms/gen:0.1359984092121247 - timing_per_token_ms/update_actor:0.06077781439001668 - perf/total_num_tokens:214936 - perf/time_per_step:35.8843690212816 - perf/throughput:748.710392094849 (TaskRunner pid=2224264) Training Progress: 20%|██ | 20/100 [18:19<1:58:06, 88.58s/it] (TaskRunner pid=2224264) step:21 - global_seqlen/min:21817 - global_seqlen/max:33018 - global_seqlen/minmax_diff:11201 - global_seqlen/balanced_min:27703 - global_seqlen/balanced_max:27729 - global_seqlen/mean:27721.25 - actor/entropy:0.6702685952186584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163779854774475 - training/rollout_probs_diff_mean:0.004457509610801935 - training/rollout_probs_diff_std:0.00877729244530201 - training/rollout_actor_probs_pearson_corr:0.9996225833892822 - rollout_corr/rollout_is_mean:1.0001065731048584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7989544868469238 - rollout_corr/rollout_is_min:0.6091647744178772 - rollout_corr/rollout_is_std:0.03465593233704567 - rollout_corr/rollout_is_eff_sample_size:0.9988006450060433 - rollout_corr/rollout_is_seq_mean:1.0000840425491333 - rollout_corr/rollout_is_seq_std:0.001599725685082376 - rollout_corr/rollout_is_seq_max:1.0058624744415283 - rollout_corr/rollout_is_seq_min:0.9920676350593567 - rollout_corr/rollout_is_seq_max_deviation:0.00793236494064331 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.009537471458316) - rollout_corr/training_log_ppl:np.float64(0.6828376608900726) - rollout_corr/kl:np.float64(0.000724996415337742) - rollout_corr/k3_kl:np.float64(0.0006654245569848172) - rollout_corr/rollout_ppl:np.float64(2.008096869569272) - rollout_corr/rollout_log_ppl:np.float64(0.6821126618888229) - rollout_corr/log_ppl_diff:np.float64(0.000724999001249671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013606392312794924) - rollout_corr/log_ppl_diff_max:np.float64(0.009048700332641602) - rollout_corr/log_ppl_diff_min:np.float64(-0.004755973815917969) - rollout_corr/ppl_ratio:np.float64(1.0007265841122717) - rollout_corr/chi2_token:np.float64(0.001218994613736868) - rollout_corr/chi2_seq:np.float64(0.7947670572903007) - actor/pg_loss:np.float64(-0.00015212141443043947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016183609022846213) - actor/ppo_kl:np.float64(0.00017459750991832834) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3394359387457371) - perf/mfu/actor:np.float64(0.10795117855645653) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.55794906616211) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003770750481635332 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003770750481635332 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:583.1015625 - response_length/max:1506.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:583.1015625 - response_length_non_aborted/max:1506.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011318549513816833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5702421497553587) - timing_s/agent_loop/generate_sequences/max:np.float64(12.702512258663774) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.579333163579577) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.702512258663774) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:1501 - timing_s/gen:18.926135268062353 - timing_s/reward:0.06378746405243874 - timing_s/old_log_prob:3.246299659833312 - timing_s/adv:0.12370392866432667 - timing_s/update_actor:13.3603654820472 - timing_s/step:35.82531424425542 - timing_s/stop_profile:0.0001258011907339096 - timing_per_token_ms/adv:0.0005578028077031459 - timing_per_token_ms/gen:0.12678788850075937 - timing_per_token_ms/update_actor:0.060244241701074085 - perf/total_num_tokens:221770 - perf/time_per_step:35.82531424425542 - perf/throughput:773.7894442738933 (TaskRunner pid=2224264) Training Progress: 21%|██ | 21/100 [18:55<1:35:47, 72.76s/it] (TaskRunner pid=2224264) step:22 - global_seqlen/min:20941 - global_seqlen/max:26032 - global_seqlen/minmax_diff:5091 - global_seqlen/balanced_min:23791 - global_seqlen/balanced_max:27438 - global_seqlen/mean:24253.875 - actor/entropy:0.6328612565994263 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18033698201179504 - training/rollout_probs_diff_mean:0.004449577536433935 - training/rollout_probs_diff_std:0.00884386245161295 - training/rollout_actor_probs_pearson_corr:0.9996188879013062 - rollout_corr/rollout_is_mean:0.9999265074729919 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.831578016281128 - rollout_corr/rollout_is_min:0.5145671367645264 - rollout_corr/rollout_is_std:0.0340801440179348 - rollout_corr/rollout_is_eff_sample_size:0.9988397723743896 - rollout_corr/rollout_is_seq_mean:0.9999299049377441 - rollout_corr/rollout_is_seq_std:0.0016637897351756692 - rollout_corr/rollout_is_seq_max:1.00577974319458 - rollout_corr/rollout_is_seq_min:0.9933801293373108 - rollout_corr/rollout_is_seq_max_deviation:0.006619870662689209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9823644189164042) - rollout_corr/training_log_ppl:np.float64(0.6632540230639279) - rollout_corr/kl:np.float64(0.0008493472823345449) - rollout_corr/k3_kl:np.float64(0.000727977444228145) - rollout_corr/rollout_ppl:np.float64(1.9806274939328432) - rollout_corr/rollout_log_ppl:np.float64(0.6624046702054329) - rollout_corr/log_ppl_diff:np.float64(0.0008493528584949672) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016184973646886647) - rollout_corr/log_ppl_diff_max:np.float64(0.007628798484802246) - rollout_corr/log_ppl_diff_min:np.float64(-0.00557255744934082) - rollout_corr/ppl_ratio:np.float64(1.000851531745866) - rollout_corr/chi2_token:np.float64(0.0012481804005801678) - rollout_corr/chi2_seq:np.float64(0.8115007842425257) - actor/pg_loss:np.float64(0.0003393040969967842) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005306760692747048) - actor/ppo_kl:np.float64(0.00014998488964579337) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.35529135540127754) - perf/mfu/actor:np.float64(0.09394398118409722) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.99365997314453) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01673314906656742 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01673314906656742 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:502.37109375 - response_length/max:4901.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:502.37109375 - response_length_non_aborted/max:4901.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013022124767303467 - timing_s/agent_loop/generate_sequences/min:np.float64(1.669778237119317) - timing_s/agent_loop/generate_sequences/max:np.float64(30.597046691924334) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.5186303148948355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.597046691924334) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:447 - timing_s/agent_loop/slowest/response_length:4901 - timing_s/gen:37.434364441782236 - timing_s/reward:0.06758200004696846 - timing_s/old_log_prob:3.2071466892957687 - timing_s/adv:0.11781474389135838 - timing_s/update_actor:13.525701014325023 - timing_s/step:54.440226985141635 - timing_s/stop_profile:0.00011406838893890381 - timing_per_token_ms/adv:0.0006071954682053815 - timing_per_token_ms/gen:0.2910756369543045 - timing_per_token_ms/update_actor:0.0697089692591649 - perf/total_num_tokens:194031 - perf/time_per_step:54.440226985141635 - perf/throughput:445.51384781366926 (TaskRunner pid=2224264) Training Progress: 22%|██▏ | 22/100 [19:49<1:27:27, 67.28s/it] (TaskRunner pid=2224264) step:23 - global_seqlen/min:20573 - global_seqlen/max:31570 - global_seqlen/minmax_diff:10997 - global_seqlen/balanced_min:28209 - global_seqlen/balanced_max:28226 - global_seqlen/mean:28218.75 - actor/entropy:0.6778538823127747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7014397382736206 - training/rollout_probs_diff_mean:0.0045140404254198074 - training/rollout_probs_diff_std:0.008873715065419674 - training/rollout_actor_probs_pearson_corr:0.9996016025543213 - rollout_corr/rollout_is_mean:0.999861478805542 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2932093341078144e-05 - rollout_corr/rollout_is_max:1.5868637561798096 - rollout_corr/rollout_is_min:0.09759196639060974 - rollout_corr/rollout_is_std:0.0343623049557209 - rollout_corr/rollout_is_eff_sample_size:0.9988203866993153 - rollout_corr/rollout_is_seq_mean:0.9999100565910339 - rollout_corr/rollout_is_seq_std:0.0015114216366782784 - rollout_corr/rollout_is_seq_max:1.004460334777832 - rollout_corr/rollout_is_seq_min:0.9952100515365601 - rollout_corr/rollout_is_seq_max_deviation:0.004789948463439941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.013377800118178) - rollout_corr/training_log_ppl:np.float64(0.6853225293452851) - rollout_corr/kl:np.float64(0.0008045039580757418) - rollout_corr/k3_kl:np.float64(0.0007065492574156451) - rollout_corr/rollout_ppl:np.float64(2.0117388791404665) - rollout_corr/rollout_log_ppl:np.float64(0.684518019144889) - rollout_corr/log_ppl_diff:np.float64(0.0008045102003961802) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014397446066141129) - rollout_corr/log_ppl_diff_max:np.float64(0.007155239582061768) - rollout_corr/log_ppl_diff_min:np.float64(-0.004554927349090576) - rollout_corr/ppl_ratio:np.float64(1.0008063388522714) - rollout_corr/chi2_token:np.float64(0.0012174241710454226) - rollout_corr/chi2_seq:np.float64(1.7728085950948298) - actor/pg_loss:np.float64(0.0003834259696304798) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000311835868842536) - actor/ppo_kl:np.float64(9.214319616290823e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32652221247553825) - perf/mfu/actor:np.float64(0.11250225764410283) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.84423065185547) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047743674367666245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047743674367666245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.1171875 - response_length/max:1391.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.1171875 - response_length_non_aborted/max:1391.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010936148464679718 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4273153580725193) - timing_s/agent_loop/generate_sequences/max:np.float64(12.947387943044305) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.094733837271633) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.947387943044305) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1391 - timing_s/gen:19.754568798467517 - timing_s/reward:0.07217162102460861 - timing_s/old_log_prob:3.125368718057871 - timing_s/adv:0.1287905927747488 - timing_s/update_actor:13.216036595404148 - timing_s/step:36.38285103067756 - timing_s/stop_profile:0.00010989978909492493 - timing_per_token_ms/adv:0.0005705009646721985 - timing_per_token_ms/gen:0.12773396613386992 - timing_per_token_ms/update_actor:0.05854279776480243 - perf/total_num_tokens:225750 - perf/time_per_step:36.38285103067756 - perf/throughput:775.6057923060046 (TaskRunner pid=2224264) Training Progress: 23%|██▎ | 23/100 [20:26<1:14:27, 58.02s/it] (TaskRunner pid=2224264) step:24 - global_seqlen/min:25634 - global_seqlen/max:31704 - global_seqlen/minmax_diff:6070 - global_seqlen/balanced_min:29137 - global_seqlen/balanced_max:29209 - global_seqlen/mean:29156.5 - actor/entropy:0.712544858455658 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3460633456707001 - training/rollout_probs_diff_mean:0.004510445520281792 - training/rollout_probs_diff_std:0.008606033399701118 - training/rollout_actor_probs_pearson_corr:0.9996117949485779 - rollout_corr/rollout_is_mean:0.9999745488166809 - rollout_corr/rollout_is_ratio_fraction_high:6.192870841914555e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.238574168382911e-05 - rollout_corr/rollout_is_max:2.087355136871338 - rollout_corr/rollout_is_min:0.47689226269721985 - rollout_corr/rollout_is_std:0.03445065766572952 - rollout_corr/rollout_is_eff_sample_size:0.9988145592508434 - rollout_corr/rollout_is_seq_mean:0.999997615814209 - rollout_corr/rollout_is_seq_std:0.0016162468818947673 - rollout_corr/rollout_is_seq_max:1.0054336786270142 - rollout_corr/rollout_is_seq_min:0.9961015582084656 - rollout_corr/rollout_is_seq_max_deviation:0.00543367862701416 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.082198773045093) - rollout_corr/training_log_ppl:np.float64(0.7211795629700646) - rollout_corr/kl:np.float64(0.0007120691560231762) - rollout_corr/k3_kl:np.float64(0.0006404822171930391) - rollout_corr/rollout_ppl:np.float64(2.0806773733347654) - rollout_corr/rollout_log_ppl:np.float64(0.7204674915410578) - rollout_corr/log_ppl_diff:np.float64(0.0007120714290067554) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014570994535461068) - rollout_corr/log_ppl_diff_max:np.float64(0.005695939064025879) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035497546195983887) - rollout_corr/ppl_ratio:np.float64(1.000713731860742) - rollout_corr/chi2_token:np.float64(0.0011395812034606934) - rollout_corr/chi2_seq:np.float64(1.328517090762034) - actor/pg_loss:np.float64(-4.311627708375454e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001907837179260241) - actor/ppo_kl:np.float64(0.0001019216878037632) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3508700765669346) - perf/mfu/actor:np.float64(0.11424736196001828) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.2998194694519) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002073063515126705 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002073063515126705 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:630.765625 - response_length/max:1901.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:630.765625 - response_length_non_aborted/max:1901.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011662021279335022 - timing_s/agent_loop/generate_sequences/min:np.float64(2.381055638194084) - timing_s/agent_loop/generate_sequences/max:np.float64(15.576675411313772) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.741719769590418) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.576675411313772) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1901 - timing_s/gen:22.96907013654709 - timing_s/reward:0.07352139614522457 - timing_s/old_log_prob:3.155397677794099 - timing_s/adv:0.11861182004213333 - timing_s/update_actor:13.56319702602923 - timing_s/step:39.965819630771875 - timing_s/stop_profile:0.00010657869279384613 - timing_per_token_ms/adv:0.0005085136249298326 - timing_per_token_ms/gen:0.14224448299776493 - timing_per_token_ms/update_actor:0.05814825607510002 - perf/total_num_tokens:233252 - perf/time_per_step:39.965819630771875 - perf/throughput:729.5358951565405 (TaskRunner pid=2224264) Training Progress: 24%|██▍ | 24/100 [21:06<1:06:38, 52.62s/it] (TaskRunner pid=2224264) step:25 - global_seqlen/min:23500 - global_seqlen/max:34274 - global_seqlen/minmax_diff:10774 - global_seqlen/balanced_min:27954 - global_seqlen/balanced_max:27980 - global_seqlen/mean:27971.25 - actor/entropy:0.7009389400482178 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18294647336006165 - training/rollout_probs_diff_mean:0.004573621321469545 - training/rollout_probs_diff_std:0.008723455481231213 - training/rollout_actor_probs_pearson_corr:0.999584436416626 - rollout_corr/rollout_is_mean:0.9999302625656128 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.836859029135667e-06 - rollout_corr/rollout_is_max:1.4950411319732666 - rollout_corr/rollout_is_min:0.4878106117248535 - rollout_corr/rollout_is_std:0.03484973683953285 - rollout_corr/rollout_is_eff_sample_size:0.9987866122287246 - rollout_corr/rollout_is_seq_mean:0.9999464154243469 - rollout_corr/rollout_is_seq_std:0.001561047974973917 - rollout_corr/rollout_is_seq_max:1.004056453704834 - rollout_corr/rollout_is_seq_min:0.9949167370796204 - rollout_corr/rollout_is_seq_max_deviation:0.005083262920379639 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1113617275841534) - rollout_corr/training_log_ppl:np.float64(0.7249491217080504) - rollout_corr/kl:np.float64(0.0006290591016089309) - rollout_corr/k3_kl:np.float64(0.0006558882103604446) - rollout_corr/rollout_ppl:np.float64(2.1101012001745403) - rollout_corr/rollout_log_ppl:np.float64(0.7243200563825667) - rollout_corr/log_ppl_diff:np.float64(0.0006290653254836798) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012821299023926258) - rollout_corr/log_ppl_diff_max:np.float64(0.005804777145385742) - rollout_corr/log_ppl_diff_min:np.float64(-0.004645586013793945) - rollout_corr/ppl_ratio:np.float64(1.0006304802373052) - rollout_corr/chi2_token:np.float64(0.00137246772646904) - rollout_corr/chi2_seq:np.float64(0.4346559832338244) - actor/pg_loss:np.float64(-0.00020399875938892365) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024524242849111033) - actor/ppo_kl:np.float64(-6.53956497700392e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31579897925257683) - perf/mfu/actor:np.float64(0.10990411377383731) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(108.17280960083008) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002174121094867587 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002174121094867587 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.3515625 - response_length/max:1596.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.3515625 - response_length_non_aborted/max:1596.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010717101395130157 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3154151998460293) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293862147256732) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.137247967235453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293862147256732) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:1596 - timing_s/gen:22.198232408612967 - timing_s/reward:0.06951954215765 - timing_s/old_log_prob:3.2000021431595087 - timing_s/adv:0.11978011205792427 - timing_s/update_actor:13.237928776070476 - timing_s/step:38.91434215568006 - timing_s/stop_profile:0.00012552551925182343 - timing_per_token_ms/adv:0.000535282263296797 - timing_per_token_ms/gen:0.15176618222015345 - timing_per_token_ms/update_actor:0.05915863956772791 - perf/total_num_tokens:223770 - perf/time_per_step:38.91434215568006 - perf/throughput:718.7902570239705 (TaskRunner pid=2224264) Training Progress: 25%|██▌ | 25/100 [21:45<1:00:38, 48.52s/it] (TaskRunner pid=2224264) step:26 - global_seqlen/min:22426 - global_seqlen/max:29951 - global_seqlen/minmax_diff:7525 - global_seqlen/balanced_min:26952 - global_seqlen/balanced_max:26964 - global_seqlen/mean:26960.375 - actor/entropy:0.7369614243507385 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862038373947144 - training/rollout_probs_diff_mean:0.004758655559271574 - training/rollout_probs_diff_std:0.00890500657260418 - training/rollout_actor_probs_pearson_corr:0.9995867609977722 - rollout_corr/rollout_is_mean:0.9999259114265442 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4305845070339274e-05 - rollout_corr/rollout_is_max:1.8682441711425781 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.03548027575016022 - rollout_corr/rollout_is_eff_sample_size:0.9987426137038081 - rollout_corr/rollout_is_seq_mean:0.9999041557312012 - rollout_corr/rollout_is_seq_std:0.001717878389172256 - rollout_corr/rollout_is_seq_max:1.005202293395996 - rollout_corr/rollout_is_seq_min:0.9952166080474854 - rollout_corr/rollout_is_seq_max_deviation:0.005202293395996094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1379281943663955) - rollout_corr/training_log_ppl:np.float64(0.7447412704350427) - rollout_corr/kl:np.float64(0.0007289938001822094) - rollout_corr/k3_kl:np.float64(0.0007074658968804215) - rollout_corr/rollout_ppl:np.float64(2.136324817314744) - rollout_corr/rollout_log_ppl:np.float64(0.7440122718689963) - rollout_corr/log_ppl_diff:np.float64(0.0007289985660463572) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015073230024427176) - rollout_corr/log_ppl_diff_max:np.float64(0.005815982818603516) - rollout_corr/log_ppl_diff_min:np.float64(-0.004653453826904297) - rollout_corr/ppl_ratio:np.float64(1.00073087005876) - rollout_corr/chi2_token:np.float64(0.0013713838998228312) - rollout_corr/chi2_seq:np.float64(0.9597642624285072) - actor/pg_loss:np.float64(0.00014449155423790216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012465850477383356) - actor/ppo_kl:np.float64(-1.70074631995476e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2908688709139824) - perf/mfu/actor:np.float64(0.105823819821739) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.64182186126709) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029881333466619253 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029881333466619253 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:546.10546875 - response_length/max:1208.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:546.10546875 - response_length_non_aborted/max:1208.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015214644372463226 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7672749813646078) - timing_s/agent_loop/generate_sequences/max:np.float64(10.418701702728868) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.990022340542055) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.418701702728868) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:1208 - timing_s/gen:18.300879038870335 - timing_s/reward:0.07066521607339382 - timing_s/old_log_prob:3.2463080435991287 - timing_s/adv:0.16146212071180344 - timing_s/update_actor:13.367159321904182 - timing_s/step:35.227726478129625 - timing_s/stop_profile:2.6691704988479614e-05 - timing_per_token_ms/adv:0.0007486084703560477 - timing_per_token_ms/gen:0.13090476627018258 - timing_per_token_ms/update_actor:0.06197595230919536 - perf/total_num_tokens:215683 - perf/time_per_step:35.227726478129625 - perf/throughput:765.3169163992961 (TaskRunner pid=2224264) Training Progress: 26%|██▌ | 26/100 [22:20<54:56, 44.54s/it] (TaskRunner pid=2224264) step:27 - global_seqlen/min:23952 - global_seqlen/max:34841 - global_seqlen/minmax_diff:10889 - global_seqlen/balanced_min:28921 - global_seqlen/balanced_max:28949 - global_seqlen/mean:28940.875 - actor/entropy:0.701714038848877 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17479756474494934 - training/rollout_probs_diff_mean:0.004477211274206638 - training/rollout_probs_diff_std:0.008556351996958256 - training/rollout_actor_probs_pearson_corr:0.9996283054351807 - rollout_corr/rollout_is_mean:0.9999887943267822 - rollout_corr/rollout_is_ratio_fraction_high:6.0401425798772834e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.0401425798772834e-06 - rollout_corr/rollout_is_max:2.401174545288086 - rollout_corr/rollout_is_min:0.40530839562416077 - rollout_corr/rollout_is_std:0.034324124455451965 - rollout_corr/rollout_is_eff_sample_size:0.9988231220557214 - rollout_corr/rollout_is_seq_mean:0.9999889135360718 - rollout_corr/rollout_is_seq_std:0.0014754357980564237 - rollout_corr/rollout_is_seq_max:1.0065122842788696 - rollout_corr/rollout_is_seq_min:0.9951512813568115 - rollout_corr/rollout_is_seq_max_deviation:0.006512284278869629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.07450102083385) - rollout_corr/training_log_ppl:np.float64(0.7144390830071643) - rollout_corr/kl:np.float64(0.0007389031823423764) - rollout_corr/k3_kl:np.float64(0.0006497853174778356) - rollout_corr/rollout_ppl:np.float64(2.0729061695747077) - rollout_corr/rollout_log_ppl:np.float64(0.713700178428553) - rollout_corr/log_ppl_diff:np.float64(0.0007389045786112547) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013226103037595749) - rollout_corr/log_ppl_diff_max:np.float64(0.0057825446128845215) - rollout_corr/log_ppl_diff_min:np.float64(-0.005777716636657715) - rollout_corr/ppl_ratio:np.float64(1.0007404030766338) - rollout_corr/chi2_token:np.float64(0.001133262412622571) - rollout_corr/chi2_seq:np.float64(0.8787905650679022) - actor/pg_loss:np.float64(-2.4800654500722885e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022700108638673555) - actor/ppo_kl:np.float64(0.00011755238243083) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32230544835329056) - perf/mfu/actor:np.float64(0.11335348091929753) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.77265930175781) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006488623563200235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006488623563200235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.71484375 - response_length/max:1758.0 - response_length/min:222.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.71484375 - response_length_non_aborted/max:1758.0 - response_length_non_aborted/min:222.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.4581090807914734e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.452285857871175) - timing_s/agent_loop/generate_sequences/max:np.float64(14.636356772854924) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.195729885730543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.636356772854924) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:194 - timing_s/agent_loop/slowest/response_length:1758 - timing_s/gen:21.458807596936822 - timing_s/reward:0.06992848217487335 - timing_s/old_log_prob:3.163326393812895 - timing_s/adv:0.12315551564097404 - timing_s/update_actor:13.37716430425644 - timing_s/step:38.279962316155434 - timing_s/stop_profile:0.00011939182877540588 - timing_per_token_ms/adv:0.0005319272293986189 - timing_per_token_ms/gen:0.1296142619666513 - timing_per_token_ms/update_actor:0.057777988330762454 - perf/total_num_tokens:231527 - perf/time_per_step:38.279962316155434 - perf/throughput:756.0319616037337 (TaskRunner pid=2224264) Training Progress: 27%|██▋ | 27/100 [22:58<51:55, 42.68s/it] (TaskRunner pid=2224264) step:28 - global_seqlen/min:25216 - global_seqlen/max:34360 - global_seqlen/minmax_diff:9144 - global_seqlen/balanced_min:29787 - global_seqlen/balanced_max:30195 - global_seqlen/mean:29843.75 - actor/entropy:0.7055036425590515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1455976366996765 - training/rollout_probs_diff_mean:0.004539164714515209 - training/rollout_probs_diff_std:0.008678900077939034 - training/rollout_actor_probs_pearson_corr:0.9996199011802673 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.916669579164591e-06 - rollout_corr/rollout_is_max:1.6682544946670532 - rollout_corr/rollout_is_min:0.4844903349876404 - rollout_corr/rollout_is_std:0.0349300317466259 - rollout_corr/rollout_is_eff_sample_size:0.9987814986875572 - rollout_corr/rollout_is_seq_mean:1.0001145601272583 - rollout_corr/rollout_is_seq_std:0.0015147364465519786 - rollout_corr/rollout_is_seq_max:1.0062413215637207 - rollout_corr/rollout_is_seq_min:0.9959177374839783 - rollout_corr/rollout_is_seq_max_deviation:0.006241321563720703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.081616838928312) - rollout_corr/training_log_ppl:np.float64(0.716681465273723) - rollout_corr/kl:np.float64(0.0007082077879232429) - rollout_corr/k3_kl:np.float64(0.0006868325160667155) - rollout_corr/rollout_ppl:np.float64(2.0801399582996964) - rollout_corr/rollout_log_ppl:np.float64(0.7159732520813122) - rollout_corr/log_ppl_diff:np.float64(0.0007082131924107671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013121537631377578) - rollout_corr/log_ppl_diff_max:np.float64(0.00504302978515625) - rollout_corr/log_ppl_diff_min:np.float64(-0.0041460394859313965) - rollout_corr/ppl_ratio:np.float64(1.0007096340414137) - rollout_corr/chi2_token:np.float64(0.0013347528874874115) - rollout_corr/chi2_seq:np.float64(4.891642920905724) - actor/pg_loss:np.float64(-6.637291517108679e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002813626040278905) - actor/ppo_kl:np.float64(0.00019132641280006357) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.33150169998407364) - perf/mfu/actor:np.float64(0.11585717187378697) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.45846176147461) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007486805785447359 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007486805785447359 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:660.2109375 - response_length/max:2519.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:660.2109375 - response_length_non_aborted/max:2519.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011588819324970245 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6954549103975296) - timing_s/agent_loop/generate_sequences/max:np.float64(18.969659883528948) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.485852991718275) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.969659883528948) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:2519 - timing_s/gen:25.569389276206493 - timing_s/reward:0.08602294698357582 - timing_s/old_log_prob:3.1977268513292074 - timing_s/adv:0.1315357405692339 - timing_s/update_actor:13.538917873054743 - timing_s/step:42.61257563531399 - timing_s/stop_profile:0.00011502020061016083 - timing_per_token_ms/adv:0.0005509350390334404 - timing_per_token_ms/gen:0.1512856288603695 - timing_per_token_ms/update_actor:0.056707509415936094 - perf/total_num_tokens:238750 - perf/time_per_step:42.61257563531399 - perf/throughput:700.3507662950987 (TaskRunner pid=2224264) Training Progress: 28%|██▊ | 28/100 [23:41<51:12, 42.67s/it] (TaskRunner pid=2224264) step:29 - global_seqlen/min:26637 - global_seqlen/max:40228 - global_seqlen/minmax_diff:13591 - global_seqlen/balanced_min:33532 - global_seqlen/balanced_max:33558 - global_seqlen/mean:33551.125 - actor/entropy:0.6751730442047119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30162957310676575 - training/rollout_probs_diff_mean:0.004225059412419796 - training/rollout_probs_diff_std:0.008393223397433758 - training/rollout_actor_probs_pearson_corr:0.9996880292892456 - rollout_corr/rollout_is_mean:0.9999781847000122 - rollout_corr/rollout_is_ratio_fraction_high:5.061727733846055e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.012345546769211e-05 - rollout_corr/rollout_is_max:2.110851764678955 - rollout_corr/rollout_is_min:0.42815104126930237 - rollout_corr/rollout_is_std:0.033269479870796204 - rollout_corr/rollout_is_eff_sample_size:0.9988943655271626 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.001319664646871388 - rollout_corr/rollout_is_seq_max:1.0051316022872925 - rollout_corr/rollout_is_seq_min:0.9961720108985901 - rollout_corr/rollout_is_seq_max_deviation:0.0051316022872924805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.010434144642204) - rollout_corr/training_log_ppl:np.float64(0.6835667735431343) - rollout_corr/kl:np.float64(0.0005298591240814332) - rollout_corr/k3_kl:np.float64(0.0006146891930143283) - rollout_corr/rollout_ppl:np.float64(2.009353395085782) - rollout_corr/rollout_log_ppl:np.float64(0.6830369132512715) - rollout_corr/log_ppl_diff:np.float64(0.0005298602918628603) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011505558213684708) - rollout_corr/log_ppl_diff_max:np.float64(0.0041373372077941895) - rollout_corr/log_ppl_diff_min:np.float64(-0.005634099245071411) - rollout_corr/ppl_ratio:np.float64(1.0005309600383043) - rollout_corr/chi2_token:np.float64(0.0014236378483474255) - rollout_corr/chi2_seq:np.float64(0.9517913018353283) - actor/pg_loss:np.float64(7.968046702444553e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001298733541261754) - actor/ppo_kl:np.float64(-2.9945098688699545e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21987130120396614) - perf/mfu/actor:np.float64(0.11725733884439407) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(109.95436000823975) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00392979895696044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00392979895696044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:771.72265625 - response_length/max:1889.0 - response_length/min:208.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:771.72265625 - response_length_non_aborted/max:1889.0 - response_length_non_aborted/min:208.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.993037045001984e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6769527215510607) - timing_s/agent_loop/generate_sequences/max:np.float64(16.69622828066349) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.844669617952604) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.69622828066349) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:1889 - timing_s/gen:23.39403291605413 - timing_s/reward:0.07856784015893936 - timing_s/old_log_prob:4.369897283613682 - timing_s/adv:0.31469788402318954 - timing_s/update_actor:15.082241535186768 - timing_s/step:43.32600663602352 - timing_s/stop_profile:0.0001105237752199173 - timing_per_token_ms/adv:0.001172456527252028 - timing_per_token_ms/gen:0.11841422606715966 - timing_per_token_ms/update_actor:0.05619126607225081 - perf/total_num_tokens:268409 - perf/time_per_step:43.32600663602352 - perf/throughput:774.387662400066 (TaskRunner pid=2224264) Training Progress: 29%|██▉ | 29/100 [24:24<50:44, 42.88s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) To determine which of the given options is the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure and functional groups involved. (TaskRunner pid=2224264) (TaskRunner pid=2224264) The target product is: (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This can be interpreted as: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **nitrile group (–CN)** attached to a ring (likely a pyrrole or similar heterocycle). (TaskRunner pid=2224264) - The ring has a **chloro group (–Cl)** on a carbon and a **trifluoromethyl group (–CF₃)** on another carbon. (TaskRunner pid=2224264) - The structure suggests that the molecule is a **pyrrole derivative** with a **chloro-substituted benzene ring** and a **trifluoromethyl group** attached to the same ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now, let's examine the options one by one to see which one could be a **reactant** that leads to this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a **complex molecule**. It contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)** and a **nitrile group (–CN)**. (TaskRunner pid=2224264) - A **benzene ring** with a **chloro group (–Cl)** and a **trifluoromethyl group (–CF₃)**. (TaskRunner pid=2224264) - A **pyrrole-like ring** (as suggested by the nitrile group and the ring structure). (TaskRunner pid=2224264) - The **nitrile group** is attached to the ring in a way that matches the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This option **matches the structure** of the target molecule. It is **likely the correct reactant** that could be used in the synthesis of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)**. (TaskRunner pid=2224264) - A **chloro group (–Cl)**. (TaskRunner pid=2224264) - A **pyrrole-like ring**. (TaskRunner pid=2224264) - However, the **trifluoromethyl group (–CF₃)** is **missing**. (TaskRunner pid=2224264) - The **nitrile group (–CN)** is **not clearly present**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule is **not a good match** for the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)**. (TaskRunner pid=2224264) - A **trifluoromethyl group (–CF₃)**. (TaskRunner pid=2224264) - A **benzene ring** with a **chloro group (–Cl)**. (TaskRunner pid=2224264) - A **pyrrole-like ring**. (TaskRunner pid=2224264) - However, the **nitrile group (–CN)** is **missing**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule is **not a good match** for the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)**. (TaskRunner pid=2224264) - A **trifluoromethyl group (–CF₃)**. (TaskRunner pid=2224264) - A **benzene ring**. (TaskRunner pid=2224264) - A **pyrrole-like ring**. (TaskRunner pid=2224264) - However, the **nitrile group (–CN)** is **missing**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Again, this molecule is **not a good match** for the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Analysis: (TaskRunner pid=2224264) Only **Option A** contains all the necessary functional groups and structure to be a **valid reactant** for the synthesis of the target molecule **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2224264) step:30 - global_seqlen/min:28624 - global_seqlen/max:34380 - global_seqlen/minmax_diff:5756 - global_seqlen/balanced_min:31823 - global_seqlen/balanced_max:31938 - global_seqlen/mean:31860.75 - actor/entropy:0.7115141153335571 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5642241835594177 - training/rollout_probs_diff_mean:0.004424138925969601 - training/rollout_probs_diff_std:0.008687857538461685 - training/rollout_actor_probs_pearson_corr:0.999598503112793 - rollout_corr/rollout_is_mean:0.9999087452888489 - rollout_corr/rollout_is_ratio_fraction_high:5.690936632163357e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.1381873264326714e-05 - rollout_corr/rollout_is_max:2.0644965171813965 - rollout_corr/rollout_is_min:0.22554616630077362 - rollout_corr/rollout_is_std:0.034556034952402115 - rollout_corr/rollout_is_eff_sample_size:0.9988071858420651 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014933171914890409 - rollout_corr/rollout_is_seq_max:1.0051254034042358 - rollout_corr/rollout_is_seq_min:0.9926964044570923 - rollout_corr/rollout_is_seq_max_deviation:0.007303595542907715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1091170106083155) - rollout_corr/training_log_ppl:np.float64(0.7282795129576698) - rollout_corr/kl:np.float64(0.0007562485593410884) - rollout_corr/k3_kl:np.float64(0.0006585642515801737) - rollout_corr/rollout_ppl:np.float64(2.107493511401117) - rollout_corr/rollout_log_ppl:np.float64(0.7275232608662918) - rollout_corr/log_ppl_diff:np.float64(0.0007562520913779736) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013768195640295744) - rollout_corr/log_ppl_diff_max:np.float64(0.005323231220245361) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032753348350524902) - rollout_corr/ppl_ratio:np.float64(1.0007577680516988) - rollout_corr/chi2_token:np.float64(0.0011402126401662827) - rollout_corr/chi2_seq:np.float64(0.9514149494934827) - actor/pg_loss:np.float64(-0.00013444386422634125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002210548966559145) - actor/ppo_kl:np.float64(4.4027712672978225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2887660190463066) - perf/mfu/actor:np.float64(0.12035666503727542) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(107.33295822143555) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.549404761904762) - val-aux/sciknoweval/reward/std@16:np.float64(0.31608937376624846) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6822666666666667) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.25471779568655717) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41630476190476196) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.28182671788474917) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5503333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3158685881058942) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7810047619047619) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1712884300404119) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.30183333333333334) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.21622513075595678) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.587) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2597657274415112) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8442666666666666) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10407977320648662) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.21562857142857142) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.14281142327250002) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.609352380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1968004873596571) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8824571428571428) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.056375240990812564) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.16108571428571428) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07902405626412515) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6248190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1453890445797554) - val-aux/sciknoweval/score/mean@16:np.float64(0.549404761904762) - val-aux/sciknoweval/score/std@16:np.float64(0.31608937376624846) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6822666666666667) - val-aux/sciknoweval/score/best@2/std:np.float64(0.25471779568655717) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.41630476190476196) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.28182671788474917) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5503333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3158685881058942) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7810047619047619) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1712884300404119) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.30183333333333334) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.21622513075595678) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.587) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2597657274415112) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8442666666666666) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10407977320648662) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.21562857142857142) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.14281142327250002) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.609352380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1968004873596571) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8824571428571428) - val-aux/sciknoweval/score/best@16/std:np.float64(0.056375240990812564) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.16108571428571428) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07902405626412515) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6248190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1453890445797554) - val-core/sciknoweval/acc/mean@16:np.float64(0.549404761904762) - val-aux/sciknoweval/acc/std@16:np.float64(0.31608937376624846) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6822666666666667) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.25471779568655717) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.41630476190476196) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.28182671788474917) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5503333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3158685881058942) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7810047619047619) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1712884300404119) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.30183333333333334) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.21622513075595678) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.587) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2597657274415112) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8442666666666666) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10407977320648662) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.21562857142857142) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.14281142327250002) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.609352380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1968004873596571) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8824571428571428) - val-core/sciknoweval/acc/best@16/std:np.float64(0.056375240990812564) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.16108571428571428) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07902405626412515) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6248190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1453890445797554) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9892857142857143) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03624438335470682) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989476190476191) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010879631942208362) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9796380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04790227517330985) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9895285714285714) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03580241391826494) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0001505093393264677) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9626380952380952) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05990016890311522) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9957904761904761) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0229613769030037) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9365666666666667) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06682360250957863) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989619047619047) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011013375581110648) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9040238095238093) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.061018043502578855) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999142857142858) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0015394818506189996) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.41796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002168958308175206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002168958308175206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:686.3984375 - response_length/max:2205.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:686.3984375 - response_length_non_aborted/max:2205.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.668455898761749e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4918374791741371) - timing_s/agent_loop/generate_sequences/max:np.float64(17.408627146854997) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.319493488706939) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.408627146854997) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:349 - timing_s/agent_loop/slowest/response_length:2205 - timing_s/gen:24.27543739043176 - timing_s/reward:0.07385276816785336 - timing_s/old_log_prob:3.287810454145074 - timing_s/adv:0.10759958252310753 - timing_s/update_actor:13.824590772390366 - timing_s/step:41.648208467289805 - timing_s/testing:186.15690745972097 - timing_s/save_checkpoint:15.06649879179895 - timing_s/stop_profile:0.00015422143042087555 - timing_per_token_ms/adv:0.00042214787208048904 - timing_per_token_ms/gen:0.13814997547452032 - timing_per_token_ms/update_actor:0.05423832918398957 - perf/total_num_tokens:254886 - perf/time_per_step:41.648208467289805 - perf/throughput:764.9968911633546 (TaskRunner pid=2224264) Training Progress: 30%|███ | 30/100 [28:27<2:00:03, 102.91s/it] (TaskRunner pid=2224264) step:31 - global_seqlen/min:30423 - global_seqlen/max:35721 - global_seqlen/minmax_diff:5298 - global_seqlen/balanced_min:33243 - global_seqlen/balanced_max:33352 - global_seqlen/mean:33261.875 - actor/entropy:0.7014114856719971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39983245730400085 - training/rollout_probs_diff_mean:0.004444092512130737 - training/rollout_probs_diff_std:0.008645395748317242 - training/rollout_actor_probs_pearson_corr:0.9995979070663452 - rollout_corr/rollout_is_mean:1.0000203847885132 - rollout_corr/rollout_is_ratio_fraction_high:5.1425781748548616e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.1425781748548616e-06 - rollout_corr/rollout_is_max:3.193486452102661 - rollout_corr/rollout_is_min:0.4723663926124573 - rollout_corr/rollout_is_std:0.03425285220146179 - rollout_corr/rollout_is_eff_sample_size:0.998828117093035 - rollout_corr/rollout_is_seq_mean:1.0000666379928589 - rollout_corr/rollout_is_seq_std:0.0014698212035000324 - rollout_corr/rollout_is_seq_max:1.0053932666778564 - rollout_corr/rollout_is_seq_min:0.9955719709396362 - rollout_corr/rollout_is_seq_max_deviation:0.005393266677856445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.08949265582487) - rollout_corr/training_log_ppl:np.float64(0.7218374547082931) - rollout_corr/kl:np.float64(0.0005589085460151466) - rollout_corr/k3_kl:np.float64(0.0006561674455269895) - rollout_corr/rollout_ppl:np.float64(2.088360784109682) - rollout_corr/rollout_log_ppl:np.float64(0.7212785450974479) - rollout_corr/log_ppl_diff:np.float64(0.0005589096108451486) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012190231354907155) - rollout_corr/log_ppl_diff_max:np.float64(0.004900336265563965) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036498308181762695) - rollout_corr/ppl_ratio:np.float64(1.0005601237062365) - rollout_corr/chi2_token:np.float64(0.0015174625441432) - rollout_corr/chi2_seq:np.float64(2.2517903256230056) - actor/pg_loss:np.float64(-9.307335130870342e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020832591962971492) - actor/ppo_kl:np.float64(1.1285746243316908e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2606422230601311) - perf/mfu/actor:np.float64(0.12484098624463014) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(113.83588361740112) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009717543609440327 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009717543609440327 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.58984375 - response_length/max:2514.0 - response_length/min:234.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.58984375 - response_length_non_aborted/max:2514.0 - response_length_non_aborted/min:234.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014492124319076538 - timing_s/agent_loop/generate_sequences/min:np.float64(3.011936265975237) - timing_s/agent_loop/generate_sequences/max:np.float64(20.60571327060461) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.018827380772564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.60571327060461) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:2514 - timing_s/gen:27.48937368579209 - timing_s/reward:0.07477623783051968 - timing_s/old_log_prob:3.7115073055028915 - timing_s/adv:0.17105644941329956 - timing_s/update_actor:13.997227132320404 - timing_s/step:45.54523225687444 - timing_s/stop_profile:0.00011219456791877747 - timing_per_token_ms/adv:0.0006428397730633779 - timing_per_token_ms/gen:0.14136624764491573 - timing_per_token_ms/update_actor:0.05260236807275749 - perf/total_num_tokens:266095 - perf/time_per_step:45.54523225687444 - perf/throughput:730.3042130162717 (TaskRunner pid=2224264) Training Progress: 31%|███ | 31/100 [29:13<1:38:34, 85.71s/it] (TaskRunner pid=2224264) step:32 - global_seqlen/min:25482 - global_seqlen/max:38028 - global_seqlen/minmax_diff:12546 - global_seqlen/balanced_min:31743 - global_seqlen/balanced_max:31780 - global_seqlen/mean:31769.5 - actor/entropy:0.6990305185317993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2773066759109497 - training/rollout_probs_diff_mean:0.004490962252020836 - training/rollout_probs_diff_std:0.00860312208533287 - training/rollout_actor_probs_pearson_corr:0.9996287226676941 - rollout_corr/rollout_is_mean:1.0000104904174805 - rollout_corr/rollout_is_ratio_fraction_high:1.0702741747081745e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.417339324951172 - rollout_corr/rollout_is_min:0.6783297061920166 - rollout_corr/rollout_is_std:0.03459395840764046 - rollout_corr/rollout_is_eff_sample_size:0.9988045694973763 - rollout_corr/rollout_is_seq_mean:0.9999669790267944 - rollout_corr/rollout_is_seq_std:0.0015490534715354443 - rollout_corr/rollout_is_seq_max:1.004462480545044 - rollout_corr/rollout_is_seq_min:0.9947811365127563 - rollout_corr/rollout_is_seq_max_deviation:0.005218863487243652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0522580044344068) - rollout_corr/training_log_ppl:np.float64(0.7055484902230091) - rollout_corr/kl:np.float64(0.0007327849301645983) - rollout_corr/k3_kl:np.float64(0.0006532650920689775) - rollout_corr/rollout_ppl:np.float64(2.0508041838184) - rollout_corr/rollout_log_ppl:np.float64(0.7048157050157897) - rollout_corr/log_ppl_diff:np.float64(0.0007327852072194219) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013509184354916215) - rollout_corr/log_ppl_diff_max:np.float64(0.006094217300415039) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032320618629455566) - rollout_corr/ppl_ratio:np.float64(1.0007343096658587) - rollout_corr/chi2_token:np.float64(0.0011649802327156067) - rollout_corr/chi2_seq:np.float64(0.8535998552106321) - actor/pg_loss:np.float64(-7.09133455529809e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026227998409922293) - actor/ppo_kl:np.float64(7.257984136543882e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28863443806767464) - perf/mfu/actor:np.float64(0.12142939169953111) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(106.63142013549805) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010431160219013691 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010431160219013691 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:729.953125 - response_length/max:1851.0 - response_length/min:205.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:729.953125 - response_length_non_aborted/max:1851.0 - response_length_non_aborted/min:205.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944157838821411 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5309949908405542) - timing_s/agent_loop/generate_sequences/max:np.float64(15.264747710898519) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.378220335514925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.264747710898519) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:1851 - timing_s/gen:22.15535177104175 - timing_s/reward:0.06814277172088623 - timing_s/old_log_prob:3.451232196763158 - timing_s/adv:0.157595532014966 - timing_s/update_actor:13.804202174767852 - timing_s/step:39.72742600180209 - timing_s/stop_profile:0.00011063739657402039 - timing_per_token_ms/adv:0.0006200740175914242 - timing_per_token_ms/gen:0.11856150743327777 - timing_per_token_ms/update_actor:0.054313894516627 - perf/total_num_tokens:254156 - perf/time_per_step:39.72742600180209 - perf/throughput:799.6868460231703 (TaskRunner pid=2224264) Training Progress: 32%|███▏ | 32/100 [29:53<1:21:31, 71.93s/it] (TaskRunner pid=2224264) step:33 - global_seqlen/min:26534 - global_seqlen/max:38525 - global_seqlen/minmax_diff:11991 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:31349 - global_seqlen/mean:31335.875 - actor/entropy:0.7530014514923096 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40599530935287476 - training/rollout_probs_diff_mean:0.004578310996294022 - training/rollout_probs_diff_std:0.00858127698302269 - training/rollout_actor_probs_pearson_corr:0.9995956420898438 - rollout_corr/rollout_is_mean:0.9998608231544495 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.69842768527451e-06 - rollout_corr/rollout_is_max:1.6179550886154175 - rollout_corr/rollout_is_min:0.4878464341163635 - rollout_corr/rollout_is_std:0.03484460711479187 - rollout_corr/rollout_is_eff_sample_size:0.9987870879096347 - rollout_corr/rollout_is_seq_mean:0.9998835921287537 - rollout_corr/rollout_is_seq_std:0.0015449043130502105 - rollout_corr/rollout_is_seq_max:1.003616213798523 - rollout_corr/rollout_is_seq_min:0.9941043257713318 - rollout_corr/rollout_is_seq_max_deviation:0.005895674228668213 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1979672331362963) - rollout_corr/training_log_ppl:np.float64(0.7705774857313372) - rollout_corr/kl:np.float64(0.0007750543829923018) - rollout_corr/k3_kl:np.float64(0.0006546921346171075) - rollout_corr/rollout_ppl:np.float64(2.1962045463733375) - rollout_corr/rollout_log_ppl:np.float64(0.7698024294222705) - rollout_corr/log_ppl_diff:np.float64(0.0007750563090667129) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013818222796544433) - rollout_corr/log_ppl_diff_max:np.float64(0.006634533405303955) - rollout_corr/log_ppl_diff_min:np.float64(-0.004233121871948242) - rollout_corr/ppl_ratio:np.float64(1.0007766336202621) - rollout_corr/chi2_token:np.float64(0.0010767022613435984) - rollout_corr/chi2_seq:np.float64(0.8498944872990251) - actor/pg_loss:np.float64(-0.0003114178543910384) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014040332393960853) - actor/ppo_kl:np.float64(3.355153801543054e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2854798920452595) - perf/mfu/actor:np.float64(0.12016219905694546) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.70193099975586) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013369936496019363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013369936496019363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:685.49609375 - response_length/max:1829.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:685.49609375 - response_length_non_aborted/max:1829.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012979470193386078 - timing_s/agent_loop/generate_sequences/min:np.float64(2.423111606389284) - timing_s/agent_loop/generate_sequences/max:np.float64(15.327023109421134) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.73543391644489) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.327023109421134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1829 - timing_s/gen:22.37846451252699 - timing_s/reward:0.07666251063346863 - timing_s/old_log_prob:3.2687183786183596 - timing_s/adv:0.12963519990444183 - timing_s/update_actor:13.795639680698514 - timing_s/step:39.736858043819666 - timing_s/stop_profile:0.00010845810174942017 - timing_per_token_ms/adv:0.0005171197545323125 - timing_per_token_ms/gen:0.12752206438384034 - timing_per_token_ms/update_actor:0.05503133262075223 - perf/total_num_tokens:250687 - perf/time_per_step:39.736858043819666 - perf/throughput:788.5846174713785 (TaskRunner pid=2224264) Training Progress: 33%|███▎ | 33/100 [30:33<1:09:33, 62.29s/it] (TaskRunner pid=2224264) step:34 - global_seqlen/min:31672 - global_seqlen/max:34621 - global_seqlen/minmax_diff:2949 - global_seqlen/balanced_min:33347 - global_seqlen/balanced_max:33381 - global_seqlen/mean:33365.5 - actor/entropy:0.7147052884101868 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26073455810546875 - training/rollout_probs_diff_mean:0.004393857438117266 - training/rollout_probs_diff_std:0.008422989398241043 - training/rollout_actor_probs_pearson_corr:0.9996627569198608 - rollout_corr/rollout_is_mean:1.0000426769256592 - rollout_corr/rollout_is_ratio_fraction_high:1.0432097042212263e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:5.339309215545654 - rollout_corr/rollout_is_min:0.5086534023284912 - rollout_corr/rollout_is_std:0.03433627635240555 - rollout_corr/rollout_is_eff_sample_size:0.99882252741175 - rollout_corr/rollout_is_seq_mean:1.0001329183578491 - rollout_corr/rollout_is_seq_std:0.001656638691201806 - rollout_corr/rollout_is_seq_max:1.0120186805725098 - rollout_corr/rollout_is_seq_min:0.9953494668006897 - rollout_corr/rollout_is_seq_max_deviation:0.012018680572509766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.108749763574451) - rollout_corr/training_log_ppl:np.float64(0.7289996044710279) - rollout_corr/kl:np.float64(0.0005081289243759057) - rollout_corr/k3_kl:np.float64(0.0006798729952492977) - rollout_corr/rollout_ppl:np.float64(2.107720549684018) - rollout_corr/rollout_log_ppl:np.float64(0.7284914773190394) - rollout_corr/log_ppl_diff:np.float64(0.0005081271519884467) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012719499645754695) - rollout_corr/log_ppl_diff_max:np.float64(0.0048035383224487305) - rollout_corr/log_ppl_diff_min:np.float64(-0.006281495094299316) - rollout_corr/ppl_ratio:np.float64(1.0005095447413623) - rollout_corr/chi2_token:np.float64(0.0017949678003787994) - rollout_corr/chi2_seq:np.float64(4.14957369142212) - actor/pg_loss:np.float64(-5.571101792156696e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020079502564840368) - actor/ppo_kl:np.float64(8.411469075042532e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3010113313794136) - perf/mfu/actor:np.float64(0.12801561879083337) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.21452951431274) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007644771598279476 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007644771598279476 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:748.890625 - response_length/max:2008.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:748.890625 - response_length_non_aborted/max:2008.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014275126159191132 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1188270319253206) - timing_s/agent_loop/generate_sequences/max:np.float64(16.865083750337362) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.53478015192377) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.865083750337362) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:201 - timing_s/agent_loop/slowest/response_length:2008 - timing_s/gen:23.755706263706088 - timing_s/reward:0.08198409341275692 - timing_s/old_log_prob:3.3104164469987154 - timing_s/adv:0.14668767154216766 - timing_s/update_actor:13.706282814964652 - timing_s/step:41.08737375959754 - timing_s/stop_profile:0.00011901184916496277 - timing_per_token_ms/adv:0.0005495484540249946 - timing_per_token_ms/gen:0.12391092169514327 - timing_per_token_ms/update_actor:0.05134900876266148 - perf/total_num_tokens:266924 - perf/time_per_step:41.08737375959754 - perf/throughput:812.0621238831601 (TaskRunner pid=2224264) Training Progress: 34%|███▍ | 34/100 [31:14<1:01:32, 55.95s/it] (TaskRunner pid=2224264) step:35 - global_seqlen/min:28121 - global_seqlen/max:32901 - global_seqlen/minmax_diff:4780 - global_seqlen/balanced_min:30268 - global_seqlen/balanced_max:30287 - global_seqlen/mean:30280.375 - actor/entropy:0.7235326170921326 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21305981278419495 - training/rollout_probs_diff_mean:0.00439543928951025 - training/rollout_probs_diff_std:0.00852300226688385 - training/rollout_actor_probs_pearson_corr:0.9996048808097839 - rollout_corr/rollout_is_mean:1.0000625848770142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.8470300245971885e-06 - rollout_corr/rollout_is_max:1.7555011510849 - rollout_corr/rollout_is_min:0.4614633619785309 - rollout_corr/rollout_is_std:0.03465937077999115 - rollout_corr/rollout_is_eff_sample_size:0.9988002882356332 - rollout_corr/rollout_is_seq_mean:1.0000625848770142 - rollout_corr/rollout_is_seq_std:0.001464530942030251 - rollout_corr/rollout_is_seq_max:1.0040720701217651 - rollout_corr/rollout_is_seq_min:0.9944659471511841 - rollout_corr/rollout_is_seq_max_deviation:0.005534052848815918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0981541997753084) - rollout_corr/training_log_ppl:np.float64(0.7273534100968391) - rollout_corr/kl:np.float64(0.0005946585815870264) - rollout_corr/k3_kl:np.float64(0.0006194554455305479) - rollout_corr/rollout_ppl:np.float64(2.0969019741751254) - rollout_corr/rollout_log_ppl:np.float64(0.7267587499227375) - rollout_corr/log_ppl_diff:np.float64(0.0005946601741015911) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012836407404392958) - rollout_corr/log_ppl_diff_max:np.float64(0.005986064672470093) - rollout_corr/log_ppl_diff_min:np.float64(-0.003438234329223633) - rollout_corr/ppl_ratio:np.float64(1.000595951685682) - rollout_corr/chi2_token:np.float64(0.001301520736888051) - rollout_corr/chi2_seq:np.float64(1.3097328948788345) - actor/pg_loss:np.float64(0.00015541678294539452) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001370309196317976) - actor/ppo_kl:np.float64(4.7729094319493015e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2635045759379864) - perf/mfu/actor:np.float64(0.11788960362270617) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.97077560424805) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020808118861168623 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020808118861168623 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:668.07421875 - response_length/max:1568.0 - response_length/min:256.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:668.07421875 - response_length_non_aborted/max:1568.0 - response_length_non_aborted/min:256.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001488998532295227 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3336482755839825) - timing_s/agent_loop/generate_sequences/max:np.float64(13.82805041782558) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.910963898211776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.82805041782558) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:1568 - timing_s/gen:20.717208752408624 - timing_s/reward:0.07455052435398102 - timing_s/old_log_prob:3.3136789835989475 - timing_s/adv:0.15318860113620758 - timing_s/update_actor:13.571898808702826 - timing_s/step:37.9156019538641 - timing_s/stop_profile:0.00011204928159713745 - timing_per_token_ms/adv:0.0006323757596141378 - timing_per_token_ms/gen:0.12113414111461128 - timing_per_token_ms/update_actor:0.056025969000973515 - perf/total_num_tokens:242243 - perf/time_per_step:37.9156019538641 - perf/throughput:798.6257223832373 (TaskRunner pid=2224264) Training Progress: 35%|███▌ | 35/100 [31:52<54:46, 50.56s/it] (TaskRunner pid=2224264) step:36 - global_seqlen/min:23484 - global_seqlen/max:37627 - global_seqlen/minmax_diff:14143 - global_seqlen/balanced_min:32177 - global_seqlen/balanced_max:32196 - global_seqlen/mean:32190.375 - actor/entropy:0.7190540432929993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9983694553375244 - training/rollout_probs_diff_mean:0.0044546606950461864 - training/rollout_probs_diff_std:0.008799256756901741 - training/rollout_actor_probs_pearson_corr:0.999652624130249 - rollout_corr/rollout_is_mean:0.9999595880508423 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.36610241397284e-06 - rollout_corr/rollout_is_max:1.631447434425354 - rollout_corr/rollout_is_min:0.00012358058302197605 - rollout_corr/rollout_is_std:0.03424588963389397 - rollout_corr/rollout_is_eff_sample_size:0.9988282360231038 - rollout_corr/rollout_is_seq_mean:0.9999497532844543 - rollout_corr/rollout_is_seq_std:0.0013012057170271873 - rollout_corr/rollout_is_seq_max:1.005060076713562 - rollout_corr/rollout_is_seq_min:0.9961336851119995 - rollout_corr/rollout_is_seq_max_deviation:0.005060076713562012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0887983799912035) - rollout_corr/training_log_ppl:np.float64(0.7234805303742178) - rollout_corr/kl:np.float64(0.0007222782085847257) - rollout_corr/k3_kl:np.float64(0.000665171026241751) - rollout_corr/rollout_ppl:np.float64(2.0872840681113303) - rollout_corr/rollout_log_ppl:np.float64(0.7227582528139465) - rollout_corr/log_ppl_diff:np.float64(0.0007222775602713227) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800676049664617) - rollout_corr/log_ppl_diff_max:np.float64(0.014325737953186035) - rollout_corr/log_ppl_diff_min:np.float64(-0.004422605037689209) - rollout_corr/ppl_ratio:np.float64(1.0007238842081279) - rollout_corr/chi2_token:np.float64(0.001139611704275012) - rollout_corr/chi2_seq:np.float64(2.33724519261159) - actor/pg_loss:np.float64(7.578812073916197e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016148608483490534) - actor/ppo_kl:np.float64(3.428134950134165e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3038923889398575) - perf/mfu/actor:np.float64(0.12476615328489614) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.85765075683594) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017614230746403337 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017614230746403337 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:727.94921875 - response_length/max:1574.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:727.94921875 - response_length_non_aborted/max:1574.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010665319859981537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2873131558299065) - timing_s/agent_loop/generate_sequences/max:np.float64(14.295806081965566) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.435247752218856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.295806081965566) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:1574 - timing_s/gen:20.98940148949623 - timing_s/reward:0.07632862776517868 - timing_s/old_log_prob:3.21400635689497 - timing_s/adv:0.11980516463518143 - timing_s/update_actor:13.706334153190255 - timing_s/step:38.190680116415024 - timing_s/stop_profile:0.0001220405101776123 - timing_per_token_ms/adv:0.0004652212215420814 - timing_per_token_ms/gen:0.11263127627107526 - timing_per_token_ms/update_actor:0.05322372818424084 - perf/total_num_tokens:257523 - perf/time_per_step:38.190680116415024 - perf/throughput:842.8856176919461 (TaskRunner pid=2224264) Training Progress: 36%|███▌ | 36/100 [32:30<49:59, 46.86s/it] (TaskRunner pid=2224264) step:37 - global_seqlen/min:27444 - global_seqlen/max:33696 - global_seqlen/minmax_diff:6252 - global_seqlen/balanced_min:31074 - global_seqlen/balanced_max:31120 - global_seqlen/mean:31108.25 - actor/entropy:0.749553382396698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1951146274805069 - training/rollout_probs_diff_mean:0.004549640696495771 - training/rollout_probs_diff_std:0.008537894114851952 - training/rollout_actor_probs_pearson_corr:0.9996463060379028 - rollout_corr/rollout_is_mean:0.9998870491981506 - rollout_corr/rollout_is_ratio_fraction_high:5.924381184740923e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0695595741271973 - rollout_corr/rollout_is_min:0.5410090088844299 - rollout_corr/rollout_is_std:0.035030562430620193 - rollout_corr/rollout_is_eff_sample_size:0.9987741257668857 - rollout_corr/rollout_is_seq_mean:0.9999449849128723 - rollout_corr/rollout_is_seq_std:0.001592667424120009 - rollout_corr/rollout_is_seq_max:1.0082616806030273 - rollout_corr/rollout_is_seq_min:0.9951016306877136 - rollout_corr/rollout_is_seq_max_deviation:0.008261680603027344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2040511020459235) - rollout_corr/training_log_ppl:np.float64(0.7697590510360897) - rollout_corr/kl:np.float64(0.0007292848855877665) - rollout_corr/k3_kl:np.float64(0.0006632836551716537) - rollout_corr/rollout_ppl:np.float64(2.202401583082974) - rollout_corr/rollout_log_ppl:np.float64(0.769029765157029) - rollout_corr/log_ppl_diff:np.float64(0.000729285879060626) - rollout_corr/log_ppl_abs_diff:np.float64(0.001381295034661889) - rollout_corr/log_ppl_diff_max:np.float64(0.005849003791809082) - rollout_corr/log_ppl_diff_min:np.float64(-0.005055129528045654) - rollout_corr/ppl_ratio:np.float64(1.0007309019565582) - rollout_corr/chi2_token:np.float64(0.001207549823448062) - rollout_corr/chi2_seq:np.float64(0.7918412457220256) - actor/pg_loss:np.float64(-0.0002318680053576827) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019654674611047085) - actor/ppo_kl:np.float64(3.777838127683175e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3199927434325218) - perf/mfu/actor:np.float64(0.11945996157211544) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.98107528686523) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011995390988886356 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011995390988886356 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:659.3515625 - response_length/max:2064.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:659.3515625 - response_length_non_aborted/max:2064.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015878677368164062 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0059274714440107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.902138613164425) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.578634077712195) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.902138613164425) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:171 - timing_s/agent_loop/slowest/response_length:2064 - timing_s/gen:23.681104212999344 - timing_s/reward:0.08328941836953163 - timing_s/old_log_prob:3.3095543030649424 - timing_s/adv:0.13988415524363518 - timing_s/update_actor:13.724779484793544 - timing_s/step:41.0210815500468 - timing_s/stop_profile:0.0001230333000421524 - timing_per_token_ms/adv:0.0005620862441781327 - timing_per_token_ms/gen:0.14029588855646138 - timing_per_token_ms/update_actor:0.05514927505080462 - perf/total_num_tokens:248866 - perf/time_per_step:41.0210815500468 - perf/throughput:758.3478744227432 (TaskRunner pid=2224264) Training Progress: 37%|███▋ | 37/100 [33:11<47:22, 45.12s/it] (TaskRunner pid=2224264) step:38 - global_seqlen/min:22559 - global_seqlen/max:30637 - global_seqlen/minmax_diff:8078 - global_seqlen/balanced_min:27684 - global_seqlen/balanced_max:27696 - global_seqlen/mean:27690.75 - actor/entropy:0.7535861730575562 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36917543411254883 - training/rollout_probs_diff_mean:0.004689306020736694 - training/rollout_probs_diff_std:0.008696562610566616 - training/rollout_actor_probs_pearson_corr:0.9996323585510254 - rollout_corr/rollout_is_mean:1.0000420808792114 - rollout_corr/rollout_is_ratio_fraction_high:6.4868513618421275e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0322699546813965 - rollout_corr/rollout_is_min:0.5430662631988525 - rollout_corr/rollout_is_std:0.03519861027598381 - rollout_corr/rollout_is_eff_sample_size:0.9987627098464102 - rollout_corr/rollout_is_seq_mean:1.0000241994857788 - rollout_corr/rollout_is_seq_std:0.0014864748809486628 - rollout_corr/rollout_is_seq_max:1.0049471855163574 - rollout_corr/rollout_is_seq_min:0.995693564414978 - rollout_corr/rollout_is_seq_max_deviation:0.004947185516357422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1352647584863007) - rollout_corr/training_log_ppl:np.float64(0.7448823321610689) - rollout_corr/kl:np.float64(0.0005967357578882115) - rollout_corr/k3_kl:np.float64(0.0006872201306578063) - rollout_corr/rollout_ppl:np.float64(2.1338966507464647) - rollout_corr/rollout_log_ppl:np.float64(0.7442855968838558) - rollout_corr/log_ppl_diff:np.float64(0.0005967352772131562) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013833154225721955) - rollout_corr/log_ppl_diff_max:np.float64(0.0049326419830322266) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489302635192871) - rollout_corr/ppl_ratio:np.float64(1.0005982399452478) - rollout_corr/chi2_token:np.float64(0.0015692522283643484) - rollout_corr/chi2_seq:np.float64(1.3063512451481074) - actor/pg_loss:np.float64(-0.00012253155000507832) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015133037595660426) - actor/ppo_kl:np.float64(-1.1263918693771302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27262789383530617) - perf/mfu/actor:np.float64(0.10989664398239996) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.94292449951172) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031607181299477816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031607181299477816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:602.1796875 - response_length/max:1274.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:602.1796875 - response_length_non_aborted/max:1274.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.338022649288177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9933321122080088) - timing_s/agent_loop/generate_sequences/max:np.float64(11.328456040471792) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.766713866491045) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.328456040471792) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:1274 - timing_s/gen:17.93396664969623 - timing_s/reward:0.07525189034640789 - timing_s/old_log_prob:3.114859651774168 - timing_s/adv:0.12139961495995522 - timing_s/update_actor:12.985273517668247 - timing_s/step:34.3180456738919 - timing_s/stop_profile:0.00011270120739936829 - timing_per_token_ms/adv:0.0005480151989380714 - timing_per_token_ms/gen:0.11633497223430656 - timing_per_token_ms/update_actor:0.05861737907815898 - perf/total_num_tokens:221526 - perf/time_per_step:34.3180456738919 - perf/throughput:806.8859824691666 (TaskRunner pid=2224264) Training Progress: 38%|███▊ | 38/100 [33:45<43:17, 41.90s/it] (TaskRunner pid=2224264) step:39 - global_seqlen/min:24673 - global_seqlen/max:36395 - global_seqlen/minmax_diff:11722 - global_seqlen/balanced_min:29976 - global_seqlen/balanced_max:30014 - global_seqlen/mean:30001.0 - actor/entropy:0.7436038255691528 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39769014716148376 - training/rollout_probs_diff_mean:0.004428871441632509 - training/rollout_probs_diff_std:0.008462125435471535 - training/rollout_actor_probs_pearson_corr:0.9996308088302612 - rollout_corr/rollout_is_mean:0.9998753666877747 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.558502197265625 - rollout_corr/rollout_is_min:0.5394280552864075 - rollout_corr/rollout_is_std:0.03459912911057472 - rollout_corr/rollout_is_eff_sample_size:0.9988039748754949 - rollout_corr/rollout_is_seq_mean:0.9998376965522766 - rollout_corr/rollout_is_seq_std:0.0016168290749192238 - rollout_corr/rollout_is_seq_max:1.0040711164474487 - rollout_corr/rollout_is_seq_min:0.9940407276153564 - rollout_corr/rollout_is_seq_max_deviation:0.005959272384643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.18053255463019) - rollout_corr/training_log_ppl:np.float64(0.7624029302969575) - rollout_corr/kl:np.float64(0.0008376638515699142) - rollout_corr/k3_kl:np.float64(0.0006715801446830483) - rollout_corr/rollout_ppl:np.float64(2.178553344681859) - rollout_corr/rollout_log_ppl:np.float64(0.7615652666427195) - rollout_corr/log_ppl_diff:np.float64(0.0008376636542379856) - rollout_corr/log_ppl_abs_diff:np.float64(0.001551654888316989) - rollout_corr/log_ppl_diff_max:np.float64(0.00657200813293457) - rollout_corr/log_ppl_diff_min:np.float64(-0.004390835762023926) - rollout_corr/ppl_ratio:np.float64(1.0008396471384913) - rollout_corr/chi2_token:np.float64(0.0010091052390635014) - rollout_corr/chi2_seq:np.float64(2.287320557516068) - actor/pg_loss:np.float64(-9.888317435979843e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001992107047499303) - actor/ppo_kl:np.float64(5.7301584355684554e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29527168720960617) - perf/mfu/actor:np.float64(0.11556263551770317) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.95692443847656) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002959122182801366 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002959122182801366 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:655.25 - response_length/max:1746.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:655.25 - response_length_non_aborted/max:1746.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001086648553609848 - timing_s/agent_loop/generate_sequences/min:np.float64(2.099978694692254) - timing_s/agent_loop/generate_sequences/max:np.float64(14.446734165772796) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.490971767700103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.446734165772796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1746 - timing_s/gen:21.59979004599154 - timing_s/reward:0.07381308265030384 - timing_s/old_log_prob:3.1907936204224825 - timing_s/adv:0.1207788810133934 - timing_s/update_actor:13.672080148011446 - timing_s/step:38.74228004924953 - timing_s/stop_profile:0.00011139921844005585 - timing_per_token_ms/adv:0.0005032285632703635 - timing_per_token_ms/gen:0.12876639430317352 - timing_per_token_ms/update_actor:0.056965101779988356 - perf/total_num_tokens:240008 - perf/time_per_step:38.74228004924953 - perf/throughput:774.3736290652606 (TaskRunner pid=2224264) Training Progress: 39%|███▉ | 39/100 [34:24<41:38, 40.96s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a target product: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) This is a nitrogen-containing compound with a specific substituent pattern. Let's break down the structure: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - The core is a **pyrrole ring** (N connected to a carbon ring). (TaskRunner pid=2224264) - The nitrogen is attached to a **carbon chain** that leads to: (TaskRunner pid=2224264) - A **chloro-substituted benzene ring** (from "Cc2ccc(Cl)cc2C(F)(F)F") (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F) (TaskRunner pid=2224264) (TaskRunner pid=2224264) So the structure is: (TaskRunner pid=2224264) **N–C–(–CH₂–C(Cl)–C(F)(F)F–)** (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now, we are to identify the **correct reactant** that could be used in the synthesis of this compound. Let's evaluate each option. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This appears to be a **pyrrole derivative** with a **carbonyl group** (O=C) and a **trifluoromethyl group**. The structure seems to match the target product. The **"c1ccn(...)"** part resembles the pyrrole ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a **strong candidate** for the correct reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **chlorine atom** on the ring and a **trifluoromethyl** group? Not directly. The structure is more complex and does not clearly match the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This includes a **trifluoromethyl group** and a **chloro-substituted benzene**, but the **N** is not directly attached to the **chloro-substituted benzene**. The structure is more complex than the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **benzene ring**, but the **N** is not directly attached to the **chloro-substituted benzene**. The structure is also more complex than the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Evaluation: (TaskRunner pid=2224264) - **Option A** contains a **pyrrole ring** with a **chloro-substituted benzene** and a **trifluoromethyl group**, which matches the structure of the target product. (TaskRunner pid=2224264) - The other options either lack the correct substituent pattern or have more complex structures that do not align with the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### ✅ **Correct Answer: A** (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2224264) step:40 - global_seqlen/min:26780 - global_seqlen/max:38560 - global_seqlen/minmax_diff:11780 - global_seqlen/balanced_min:31398 - global_seqlen/balanced_max:31614 - global_seqlen/mean:31430.375 - actor/entropy:0.7148814797401428 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.353511780500412 - training/rollout_probs_diff_mean:0.004250292666256428 - training/rollout_probs_diff_std:0.008334573358297348 - training/rollout_actor_probs_pearson_corr:0.9996918439865112 - rollout_corr/rollout_is_mean:0.9999884963035583 - rollout_corr/rollout_is_ratio_fraction_high:5.834475814481266e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.834475814481266e-06 - rollout_corr/rollout_is_max:2.314491033554077 - rollout_corr/rollout_is_min:0.4158317446708679 - rollout_corr/rollout_is_std:0.034074895083904266 - rollout_corr/rollout_is_eff_sample_size:0.9988401291730077 - rollout_corr/rollout_is_seq_mean:0.9999082088470459 - rollout_corr/rollout_is_seq_std:0.0015310312155634165 - rollout_corr/rollout_is_seq_max:1.0046111345291138 - rollout_corr/rollout_is_seq_min:0.9931868314743042 - rollout_corr/rollout_is_seq_max_deviation:0.006813168525695801 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0881334473378956) - rollout_corr/training_log_ppl:np.float64(0.7168666583602317) - rollout_corr/kl:np.float64(0.0009168515699862212) - rollout_corr/k3_kl:np.float64(0.0007126086813400434) - rollout_corr/rollout_ppl:np.float64(2.0860950271598995) - rollout_corr/rollout_log_ppl:np.float64(0.7159498034743592) - rollout_corr/log_ppl_diff:np.float64(0.0009168548858724535) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013491588761098683) - rollout_corr/log_ppl_diff_max:np.float64(0.006253480911254883) - rollout_corr/log_ppl_diff_min:np.float64(-0.003960609436035156) - rollout_corr/ppl_ratio:np.float64(1.0009184894151986) - rollout_corr/chi2_token:np.float64(0.0010311168152838945) - rollout_corr/chi2_seq:np.float64(0.5281695865560323) - actor/pg_loss:np.float64(0.0002642440376803279) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030195120598364156) - actor/ppo_kl:np.float64(0.00023282111740385858) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26611069589853287) - perf/mfu/actor:np.float64(0.12153132959867792) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.97843170166016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6053571428571428) - val-aux/sciknoweval/reward/std@16:np.float64(0.28468084712859354) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7211619047619049) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22140257031009367) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.4908571428571428) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.25871458688057813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6054666666666666) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28424403856079056) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8044809523809524) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15305264517858658) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.3869142857142857) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.2100138722335839) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6357761904761906) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22516406707067232) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8634285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09899214217926788) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3012904761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.15568733164559595) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6521095238095237) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15983092148992065) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9010238095238096) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.056177528972713935) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.23781428571428573) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.09852453096057993) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6605380952380953) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11372781200461213) - val-aux/sciknoweval/score/mean@16:np.float64(0.6053571428571428) - val-aux/sciknoweval/score/std@16:np.float64(0.28468084712859354) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7211619047619049) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22140257031009367) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.4908571428571428) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.25871458688057813) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6054666666666666) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28424403856079056) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8044809523809524) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15305264517858658) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.3869142857142857) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.2100138722335839) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6357761904761906) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22516406707067232) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8634285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09899214217926788) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3012904761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.15568733164559595) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6521095238095237) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15983092148992065) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9010238095238096) - val-aux/sciknoweval/score/best@16/std:np.float64(0.056177528972713935) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.23781428571428573) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.09852453096057993) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6605380952380953) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11372781200461213) - val-core/sciknoweval/acc/mean@16:np.float64(0.6053571428571428) - val-aux/sciknoweval/acc/std@16:np.float64(0.28468084712859354) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7211619047619049) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22140257031009367) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.4908571428571428) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.25871458688057813) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6054666666666666) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28424403856079056) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8044809523809524) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15305264517858658) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.3869142857142857) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.2100138722335839) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6357761904761906) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22516406707067232) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8634285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09899214217926788) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3012904761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.15568733164559595) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6521095238095237) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15983092148992065) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9010238095238096) - val-core/sciknoweval/acc/best@16/std:np.float64(0.056177528972713935) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.23781428571428573) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.09852453096057993) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6605380952380953) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11372781200461213) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.986904761904762) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.036439037613581124) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9981857142857142) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0128589514254664) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9762047619047618) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04654134805312809) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9871476190476189) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036109563011539794) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999333333333333) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0014234611985593974) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9578047619047619) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05537427217213782) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9950380952380953) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.022826156373882658) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9323095238095238) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05709074252726499) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9987190476190476) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010923127179237949) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9055857142857143) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04723161312584812) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998666666666666) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002542800942671283) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004572770558297634 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004572770558297634 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:669.51171875 - response_length/max:1961.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:669.51171875 - response_length_non_aborted/max:1961.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011765025556087494 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0561186894774437) - timing_s/agent_loop/generate_sequences/max:np.float64(16.069326899945736) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.7381278641551035) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.069326899945736) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:1961 - timing_s/gen:22.699881782755256 - timing_s/reward:0.07818994671106339 - timing_s/old_log_prob:3.2518085297197104 - timing_s/adv:0.13994908705353737 - timing_s/update_actor:13.636622594669461 - timing_s/step:39.892101377248764 - timing_s/testing:188.02605186402798 - timing_s/save_checkpoint:13.08927645906806 - timing_s/stop_profile:0.00019207783043384552 - timing_per_token_ms/adv:0.0005565837468274613 - timing_per_token_ms/gen:0.1324419136074871 - timing_per_token_ms/update_actor:0.054233454877127066 - perf/total_num_tokens:251443 - perf/time_per_step:39.892101377248764 - perf/throughput:787.8846667607576 (TaskRunner pid=2224264) Training Progress: 40%|████ | 40/100 [38:25<1:41:00, 101.00s/it] (TaskRunner pid=2224264) step:41 - global_seqlen/min:25184 - global_seqlen/max:36473 - global_seqlen/minmax_diff:11289 - global_seqlen/balanced_min:30315 - global_seqlen/balanced_max:30362 - global_seqlen/mean:30349.125 - actor/entropy:0.7456910610198975 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1853349208831787 - training/rollout_probs_diff_mean:0.004585789050906897 - training/rollout_probs_diff_std:0.008587997406721115 - training/rollout_actor_probs_pearson_corr:0.9996112585067749 - rollout_corr/rollout_is_mean:1.000074863433838 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1413375432312023e-05 - rollout_corr/rollout_is_max:1.7971372604370117 - rollout_corr/rollout_is_min:0.38024628162384033 - rollout_corr/rollout_is_std:0.0351342037320137 - rollout_corr/rollout_is_eff_sample_size:0.9987673475326293 - rollout_corr/rollout_is_seq_mean:1.000067949295044 - rollout_corr/rollout_is_seq_std:0.0015586130321025848 - rollout_corr/rollout_is_seq_max:1.005883812904358 - rollout_corr/rollout_is_seq_min:0.993992805480957 - rollout_corr/rollout_is_seq_max_deviation:0.006007194519042969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1489445390179753) - rollout_corr/training_log_ppl:np.float64(0.7491948831593618) - rollout_corr/kl:np.float64(0.0007140677866042111) - rollout_corr/k3_kl:np.float64(0.000693655981081065) - rollout_corr/rollout_ppl:np.float64(2.147350816987455) - rollout_corr/rollout_log_ppl:np.float64(0.7484808203298599) - rollout_corr/log_ppl_diff:np.float64(0.0007140628295019269) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013865955406799912) - rollout_corr/log_ppl_diff_max:np.float64(0.00584721565246582) - rollout_corr/log_ppl_diff_min:np.float64(-0.004841506481170654) - rollout_corr/ppl_ratio:np.float64(1.0007157281506807) - rollout_corr/chi2_token:np.float64(0.0013592832256108522) - rollout_corr/chi2_seq:np.float64(1.1909501447807997) - actor/pg_loss:np.float64(0.000211457721889019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024292130228786846) - actor/ppo_kl:np.float64(0.00014254798117097067) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2715475298464298) - perf/mfu/actor:np.float64(0.11821237532599987) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(105.00201988220215) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00029318107408471406 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00029318107408471406 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:684.50390625 - response_length/max:1922.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:684.50390625 - response_length_non_aborted/max:1922.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.857250213623047e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0997980404645205) - timing_s/agent_loop/generate_sequences/max:np.float64(15.434562433511019) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.771742777309555) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.434562433511019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:1922 - timing_s/gen:22.271852983161807 - timing_s/reward:0.07384450361132622 - timing_s/old_log_prob:3.3064878918230534 - timing_s/adv:0.13560799323022366 - timing_s/update_actor:13.53317659534514 - timing_s/step:39.414824433624744 - timing_s/stop_profile:0.00011513754725456238 - timing_per_token_ms/adv:0.0005585333729976715 - timing_per_token_ms/gen:0.12709850874642223 - timing_per_token_ms/update_actor:0.055739566607542805 - perf/total_num_tokens:242793 - perf/time_per_step:39.414824433624744 - perf/throughput:769.9926470840549 (TaskRunner pid=2224264) Training Progress: 41%|████ | 41/100 [39:05<1:21:09, 82.54s/it] (TaskRunner pid=2224264) step:42 - global_seqlen/min:23559 - global_seqlen/max:32248 - global_seqlen/minmax_diff:8689 - global_seqlen/balanced_min:29140 - global_seqlen/balanced_max:29158 - global_seqlen/mean:29150.625 - actor/entropy:0.7418811321258545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8146259784698486 - training/rollout_probs_diff_mean:0.004636453464627266 - training/rollout_probs_diff_std:0.008857984095811844 - training/rollout_actor_probs_pearson_corr:0.9996300339698792 - rollout_corr/rollout_is_mean:0.9999567270278931 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.9698286349885166e-06 - rollout_corr/rollout_is_max:1.5637112855911255 - rollout_corr/rollout_is_min:0.1537874937057495 - rollout_corr/rollout_is_std:0.03467312455177307 - rollout_corr/rollout_is_eff_sample_size:0.9987990990027735 - rollout_corr/rollout_is_seq_mean:0.9999449849128723 - rollout_corr/rollout_is_seq_std:0.0014816136099398136 - rollout_corr/rollout_is_seq_max:1.006678581237793 - rollout_corr/rollout_is_seq_min:0.9932494759559631 - rollout_corr/rollout_is_seq_max_deviation:0.006750524044036865 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.149658269248903) - rollout_corr/training_log_ppl:np.float64(0.750017037265934) - rollout_corr/kl:np.float64(0.0007465042794017052) - rollout_corr/k3_kl:np.float64(0.0006643551571414719) - rollout_corr/rollout_ppl:np.float64(2.148035450838506) - rollout_corr/rollout_log_ppl:np.float64(0.7492705312324688) - rollout_corr/log_ppl_diff:np.float64(0.0007465060334652662) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012925199698656797) - rollout_corr/log_ppl_diff_max:np.float64(0.005987584590911865) - rollout_corr/log_ppl_diff_min:np.float64(-0.00581049919128418) - rollout_corr/ppl_ratio:np.float64(1.0007479563355446) - rollout_corr/chi2_token:np.float64(0.0011864088010042906) - rollout_corr/chi2_seq:np.float64(0.5256321090273559) - actor/pg_loss:np.float64(-3.93550144508481e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024687639711373777) - actor/ppo_kl:np.float64(6.906396298589357e-05) - actor/pg_clipfrac_lower:np.float64(8.510348379786592e-06) - actor/grad_norm:np.float64(0.31687242165207863) - perf/mfu/actor:np.float64(0.11438361389153799) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.63388013839722) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007985391654074192 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007985391654074192 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:654.33203125 - response_length/max:1379.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:654.33203125 - response_length_non_aborted/max:1379.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015127286314964294 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8712534438818693) - timing_s/agent_loop/generate_sequences/max:np.float64(12.603244848549366) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.466689185537689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.603244848549366) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1379 - timing_s/gen:20.526346927508712 - timing_s/reward:0.07452544383704662 - timing_s/old_log_prob:3.2816093750298023 - timing_s/adv:0.14070634730160236 - timing_s/update_actor:13.285876266658306 - timing_s/step:37.395900616422296 - timing_s/stop_profile:0.00011482276022434235 - timing_per_token_ms/adv:0.0006033590501987623 - timing_per_token_ms/gen:0.12253877061834714 - timing_per_token_ms/update_actor:0.056970803656260824 - perf/total_num_tokens:233205 - perf/time_per_step:37.395900616422296 - perf/throughput:779.5139178222811 (TaskRunner pid=2224264) Training Progress: 42%|████▏ | 42/100 [39:42<1:06:43, 69.02s/it] (TaskRunner pid=2224264) step:43 - global_seqlen/min:28157 - global_seqlen/max:35791 - global_seqlen/minmax_diff:7634 - global_seqlen/balanced_min:32488 - global_seqlen/balanced_max:32514 - global_seqlen/mean:32503.125 - actor/entropy:0.7455630302429199 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102365970611572 - training/rollout_probs_diff_mean:0.0044556111097335815 - training/rollout_probs_diff_std:0.008428295142948627 - training/rollout_actor_probs_pearson_corr:0.9996055960655212 - rollout_corr/rollout_is_mean:1.0000288486480713 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1921050574746914e-05 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.3978634774684906 - rollout_corr/rollout_is_std:0.03456293046474457 - rollout_corr/rollout_is_eff_sample_size:0.9988067101420113 - rollout_corr/rollout_is_seq_mean:1.000025749206543 - rollout_corr/rollout_is_seq_std:0.0014121269341558218 - rollout_corr/rollout_is_seq_max:1.0047554969787598 - rollout_corr/rollout_is_seq_min:0.9933763146400452 - rollout_corr/rollout_is_seq_max_deviation:0.006623685359954834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1632221047766507) - rollout_corr/training_log_ppl:np.float64(0.7551403474062681) - rollout_corr/kl:np.float64(0.0007046414169096238) - rollout_corr/k3_kl:np.float64(0.0006673236663345961) - rollout_corr/rollout_ppl:np.float64(2.16164415422827) - rollout_corr/rollout_log_ppl:np.float64(0.7544357032165863) - rollout_corr/log_ppl_diff:np.float64(0.0007046441896818578) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013581206439994276) - rollout_corr/log_ppl_diff_max:np.float64(0.013478577136993408) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035314559936523438) - rollout_corr/ppl_ratio:np.float64(1.0007065150421113) - rollout_corr/chi2_token:np.float64(0.0012414606753736734) - rollout_corr/chi2_seq:np.float64(1.941952116554603) - actor/pg_loss:np.float64(6.042851600795984e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017739249074111285) - actor/ppo_kl:np.float64(0.00010932787842676817) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27033093199133873) - perf/mfu/actor:np.float64(0.12426556577467283) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.3560562133789) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005291193723678589 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005291193723678589 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:712.78515625 - response_length/max:1722.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:712.78515625 - response_length_non_aborted/max:1722.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010757148265838623 - timing_s/agent_loop/generate_sequences/min:np.float64(2.216030417010188) - timing_s/agent_loop/generate_sequences/max:np.float64(14.333171149715781) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.09952915172471) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.333171149715781) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:1722 - timing_s/gen:21.01793995499611 - timing_s/reward:0.07846580259501934 - timing_s/old_log_prob:3.292416427284479 - timing_s/adv:0.14018669165670872 - timing_s/update_actor:13.660967510193586 - timing_s/step:38.27972553297877 - timing_s/stop_profile:0.00010936520993709564 - timing_per_token_ms/adv:0.0005391277440888712 - timing_per_token_ms/gen:0.11518383517011344 - timing_per_token_ms/update_actor:0.05253713108429415 - perf/total_num_tokens:260025 - perf/time_per_step:38.27972553297877 - perf/throughput:849.0950378418437 (TaskRunner pid=2224264) Training Progress: 43%|████▎ | 43/100 [40:21<56:49, 59.81s/it] (TaskRunner pid=2224264) step:44 - global_seqlen/min:28261 - global_seqlen/max:39735 - global_seqlen/minmax_diff:11474 - global_seqlen/balanced_min:33071 - global_seqlen/balanced_max:39250 - global_seqlen/mean:33860.375 - actor/entropy:0.7164559364318848 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3860778212547302 - training/rollout_probs_diff_mean:0.0043455152772367 - training/rollout_probs_diff_std:0.008481818251311779 - training/rollout_actor_probs_pearson_corr:0.9996494650840759 - rollout_corr/rollout_is_mean:1.0000731945037842 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.569669802847784e-05 - rollout_corr/rollout_is_max:1.6253478527069092 - rollout_corr/rollout_is_min:0.3264278769493103 - rollout_corr/rollout_is_std:0.03381501883268356 - rollout_corr/rollout_is_eff_sample_size:0.9988579694289216 - rollout_corr/rollout_is_seq_mean:1.0000576972961426 - rollout_corr/rollout_is_seq_std:0.0014711449621245265 - rollout_corr/rollout_is_seq_max:1.0052176713943481 - rollout_corr/rollout_is_seq_min:0.9947458505630493 - rollout_corr/rollout_is_seq_max_deviation:0.005254149436950684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1722452212125063) - rollout_corr/training_log_ppl:np.float64(0.7589113183676091) - rollout_corr/kl:np.float64(0.0006052850321509595) - rollout_corr/k3_kl:np.float64(0.0006508081007154942) - rollout_corr/rollout_ppl:np.float64(2.170979785732925) - rollout_corr/rollout_log_ppl:np.float64(0.7583060346714774) - rollout_corr/log_ppl_diff:np.float64(0.0006052836961316643) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306259231569129) - rollout_corr/log_ppl_diff_max:np.float64(0.005559146404266357) - rollout_corr/log_ppl_diff_min:np.float64(-0.00449979305267334) - rollout_corr/ppl_ratio:np.float64(1.0006066909991205) - rollout_corr/chi2_token:np.float64(0.001387621508911252) - rollout_corr/chi2_seq:np.float64(1.5838370565325022) - actor/pg_loss:np.float64(8.792639710009098e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022318573451229895) - actor/ppo_kl:np.float64(5.0495547061046864e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.303693313151598) - perf/mfu/actor:np.float64(0.121345258264149) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.35682678222656) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03973030298948288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03973030298948288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:746.57421875 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:746.57421875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.71539455652237e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3765512369573116) - timing_s/agent_loop/generate_sequences/max:np.float64(61.594920651987195) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.441452034945542) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(61.594920651987195) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:384 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:68.32033305615187 - timing_s/reward:0.07946610450744629 - timing_s/old_log_prob:3.648138355463743 - timing_s/adv:0.15895853005349636 - timing_s/update_actor:14.574448632076383 - timing_s/step:86.86667879670858 - timing_s/stop_profile:0.00011816807091236115 - timing_per_token_ms/adv:0.0005868161902131044 - timing_per_token_ms/gen:0.35746787700146954 - timing_per_token_ms/update_actor:0.0538034820644942 - perf/total_num_tokens:270883 - perf/time_per_step:86.86667879670858 - perf/throughput:389.7970484084282 (TaskRunner pid=2224264) Training Progress: 44%|████▍ | 44/100 [41:47<1:03:24, 67.94s/it] (TaskRunner pid=2224264) step:45 - global_seqlen/min:30046 - global_seqlen/max:36319 - global_seqlen/minmax_diff:6273 - global_seqlen/balanced_min:33079 - global_seqlen/balanced_max:33119 - global_seqlen/mean:33104.0 - actor/entropy:0.7044548392295837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456476628780365 - training/rollout_probs_diff_mean:0.004338444676250219 - training/rollout_probs_diff_std:0.008459679782390594 - training/rollout_actor_probs_pearson_corr:0.999611496925354 - rollout_corr/rollout_is_mean:1.0000234842300415 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.426760253612883e-06 - rollout_corr/rollout_is_max:1.7352091073989868 - rollout_corr/rollout_is_min:0.412738174200058 - rollout_corr/rollout_is_std:0.03374973684549332 - rollout_corr/rollout_is_eff_sample_size:0.9988624890615934 - rollout_corr/rollout_is_seq_mean:0.9999771118164062 - rollout_corr/rollout_is_seq_std:0.0013954972382634878 - rollout_corr/rollout_is_seq_max:1.0044217109680176 - rollout_corr/rollout_is_seq_min:0.9954630732536316 - rollout_corr/rollout_is_seq_max_deviation:0.004536926746368408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1041139638982713) - rollout_corr/training_log_ppl:np.float64(0.7206308286986314) - rollout_corr/kl:np.float64(0.0006204138266534542) - rollout_corr/k3_kl:np.float64(0.0006203657341075086) - rollout_corr/rollout_ppl:np.float64(2.102761774789542) - rollout_corr/rollout_log_ppl:np.float64(0.7200104069779627) - rollout_corr/log_ppl_diff:np.float64(0.0006204217206686735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011971264611929655) - rollout_corr/log_ppl_diff_max:np.float64(0.005265474319458008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036600232124328613) - rollout_corr/ppl_ratio:np.float64(1.0006216508336365) - rollout_corr/chi2_token:np.float64(0.0012450276408344507) - rollout_corr/chi2_seq:np.float64(1.1021782979369164) - actor/pg_loss:np.float64(-7.425120566040277e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015113081622075697) - actor/ppo_kl:np.float64(1.2432504128412347e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29480820521712303) - perf/mfu/actor:np.float64(0.1272612688514589) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.27553939819336) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005770002957433462 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005770002957433462 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:719.8125 - response_length/max:1962.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:719.8125 - response_length_non_aborted/max:1962.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.384138345718384e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.28562774322927) - timing_s/agent_loop/generate_sequences/max:np.float64(16.55846000276506) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.204205383684894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.55846000276506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:201 - timing_s/agent_loop/slowest/response_length:1962 - timing_s/gen:23.205180127173662 - timing_s/reward:0.0775289349257946 - timing_s/old_log_prob:3.335056768730283 - timing_s/adv:0.130776172503829 - timing_s/update_actor:13.470531040802598 - timing_s/step:40.30686186812818 - timing_s/stop_profile:0.00012123957276344299 - timing_per_token_ms/adv:0.0004938080462475418 - timing_per_token_ms/gen:0.1259289535424463 - timing_per_token_ms/update_actor:0.05086443874155162 - perf/total_num_tokens:264832 - perf/time_per_step:40.30686186812818 - perf/throughput:821.2993635750221 (TaskRunner pid=2224264) Training Progress: 45%|████▌ | 45/100 [42:28<54:41, 59.66s/it] (TaskRunner pid=2224264) step:46 - global_seqlen/min:25951 - global_seqlen/max:37896 - global_seqlen/minmax_diff:11945 - global_seqlen/balanced_min:31356 - global_seqlen/balanced_max:31791 - global_seqlen/mean:31428.0 - actor/entropy:0.7460911870002747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456478089094162 - training/rollout_probs_diff_mean:0.004542972426861525 - training/rollout_probs_diff_std:0.008595590479671955 - training/rollout_actor_probs_pearson_corr:0.9995923638343811 - rollout_corr/rollout_is_mean:1.0000001192092896 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1051189176214393e-05 - rollout_corr/rollout_is_max:1.62330961227417 - rollout_corr/rollout_is_min:0.39613932371139526 - rollout_corr/rollout_is_std:0.03452324494719505 - rollout_corr/rollout_is_eff_sample_size:0.9988095643491313 - rollout_corr/rollout_is_seq_mean:0.9999744892120361 - rollout_corr/rollout_is_seq_std:0.0015889799688011408 - rollout_corr/rollout_is_seq_max:1.0049422979354858 - rollout_corr/rollout_is_seq_min:0.9941216111183167 - rollout_corr/rollout_is_seq_max_deviation:0.00587838888168335 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2059524417854846) - rollout_corr/training_log_ppl:np.float64(0.7630574661307037) - rollout_corr/kl:np.float64(0.0006121348877812238) - rollout_corr/k3_kl:np.float64(0.000635629686428274) - rollout_corr/rollout_ppl:np.float64(2.204429803881794) - rollout_corr/rollout_log_ppl:np.float64(0.7624453329481184) - rollout_corr/log_ppl_diff:np.float64(0.0006121331825852394) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012938708532601595) - rollout_corr/log_ppl_diff_max:np.float64(0.006876647472381592) - rollout_corr/log_ppl_diff_min:np.float64(-0.004626870155334473) - rollout_corr/ppl_ratio:np.float64(1.0006135429721326) - rollout_corr/chi2_token:np.float64(0.0013200147077441216) - rollout_corr/chi2_seq:np.float64(1.2292231675237417) - actor/pg_loss:np.float64(-5.2339513786137104e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001717434528245576) - actor/ppo_kl:np.float64(-3.4712943731474155e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27383991330862045) - perf/mfu/actor:np.float64(0.11895889859672076) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.13016891479492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014861086383461952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014861086383461952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:706.9375 - response_length/max:2609.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:706.9375 - response_length_non_aborted/max:2609.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.24672931432724e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7639765553176403) - timing_s/agent_loop/generate_sequences/max:np.float64(19.902556966990232) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.572092931586667) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.902556966990232) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:177 - timing_s/agent_loop/slowest/response_length:2609 - timing_s/gen:27.38719269633293 - timing_s/reward:0.0768952015787363 - timing_s/old_log_prob:3.2372986152768135 - timing_s/adv:0.13518765196204185 - timing_s/update_actor:14.003670601174235 - timing_s/step:44.92860447615385 - timing_s/stop_profile:0.00011640042066574097 - timing_per_token_ms/adv:0.0005376879373569821 - timing_per_token_ms/gen:0.15133052281149395 - timing_per_token_ms/update_actor:0.055697429844303786 - perf/total_num_tokens:251424 - perf/time_per_step:44.92860447615385 - perf/throughput:699.5098193330401 (TaskRunner pid=2224264) Training Progress: 46%|████▌ | 46/100 [43:13<49:43, 55.26s/it] (TaskRunner pid=2224264) step:47 - global_seqlen/min:28796 - global_seqlen/max:40193 - global_seqlen/minmax_diff:11397 - global_seqlen/balanced_min:34771 - global_seqlen/balanced_max:34835 - global_seqlen/mean:34821.25 - actor/entropy:0.7023751139640808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34761902689933777 - training/rollout_probs_diff_mean:0.004359257873147726 - training/rollout_probs_diff_std:0.008405433967709541 - training/rollout_actor_probs_pearson_corr:0.9996216297149658 - rollout_corr/rollout_is_mean:0.9998831152915955 - rollout_corr/rollout_is_ratio_fraction_high:4.914246346743312e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.914246346743312e-06 - rollout_corr/rollout_is_max:2.393986701965332 - rollout_corr/rollout_is_min:0.404596209526062 - rollout_corr/rollout_is_std:0.03408888727426529 - rollout_corr/rollout_is_eff_sample_size:0.9988389398452722 - rollout_corr/rollout_is_seq_mean:0.9998857975006104 - rollout_corr/rollout_is_seq_std:0.0014544243458658457 - rollout_corr/rollout_is_seq_max:1.0058720111846924 - rollout_corr/rollout_is_seq_min:0.9959080219268799 - rollout_corr/rollout_is_seq_max_deviation:0.005872011184692383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.088344756513834) - rollout_corr/training_log_ppl:np.float64(0.71887747416622) - rollout_corr/kl:np.float64(0.0006769733809974809) - rollout_corr/k3_kl:np.float64(0.0006363334275647503) - rollout_corr/rollout_ppl:np.float64(2.086956723127514) - rollout_corr/rollout_log_ppl:np.float64(0.7182004956412129) - rollout_corr/log_ppl_diff:np.float64(0.0006769785250071436) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012075564882252365) - rollout_corr/log_ppl_diff_max:np.float64(0.006249547004699707) - rollout_corr/log_ppl_diff_min:np.float64(-0.004652619361877441) - rollout_corr/ppl_ratio:np.float64(1.0006782109849155) - rollout_corr/chi2_token:np.float64(0.0012049346696585417) - rollout_corr/chi2_seq:np.float64(0.8489019055850804) - actor/pg_loss:np.float64(0.00018669036217033863) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019566034120543918) - actor/ppo_kl:np.float64(-4.604769495308858e-05) - actor/pg_clipfrac_lower:np.float64(4.4389203139871825e-06) - actor/grad_norm:np.float64(0.2251530885696411) - perf/mfu/actor:np.float64(0.1320186966648972) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.04901123046875) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006613961420953274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006613961420953274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:794.8828125 - response_length/max:2347.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:794.8828125 - response_length_non_aborted/max:2347.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010444037616252899 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4994081798940897) - timing_s/agent_loop/generate_sequences/max:np.float64(18.286521969363093) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.572983121870493) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.286521969363093) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:2347 - timing_s/gen:25.14034183882177 - timing_s/reward:0.07981901802122593 - timing_s/old_log_prob:3.3178899493068457 - timing_s/adv:0.1422868575900793 - timing_s/update_actor:13.743424396961927 - timing_s/step:42.512125335633755 - timing_s/stop_profile:0.00011054240167140961 - timing_per_token_ms/adv:0.0005107759543026145 - timing_per_token_ms/gen:0.12354583438410618 - timing_per_token_ms/update_actor:0.04933562263331273 - perf/total_num_tokens:278570 - perf/time_per_step:42.512125335633755 - perf/throughput:819.0898414296109 (TaskRunner pid=2224264) Training Progress: 47%|████▋ | 47/100 [43:55<45:26, 51.45s/it] (TaskRunner pid=2224264) step:48 - global_seqlen/min:21182 - global_seqlen/max:34146 - global_seqlen/minmax_diff:12964 - global_seqlen/balanced_min:29830 - global_seqlen/balanced_max:29848 - global_seqlen/mean:29841.125 - actor/entropy:0.7442194819450378 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4606970250606537 - training/rollout_probs_diff_mean:0.004590690601617098 - training/rollout_probs_diff_std:0.008653250522911549 - training/rollout_actor_probs_pearson_corr:0.999643087387085 - rollout_corr/rollout_is_mean:0.9999340772628784 - rollout_corr/rollout_is_ratio_fraction_high:1.1716118024196476e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.565861463546753 - rollout_corr/rollout_is_min:0.6271895170211792 - rollout_corr/rollout_is_std:0.03491979092359543 - rollout_corr/rollout_is_eff_sample_size:0.9987818554445442 - rollout_corr/rollout_is_seq_mean:0.9998950362205505 - rollout_corr/rollout_is_seq_std:0.0015674727037549019 - rollout_corr/rollout_is_seq_max:1.0053834915161133 - rollout_corr/rollout_is_seq_min:0.9940774440765381 - rollout_corr/rollout_is_seq_max_deviation:0.005922555923461914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.153257429599762) - rollout_corr/training_log_ppl:np.float64(0.7468871437013149) - rollout_corr/kl:np.float64(0.0006051990699234011) - rollout_corr/k3_kl:np.float64(0.0006705579712615872) - rollout_corr/rollout_ppl:np.float64(2.1520667281001806) - rollout_corr/rollout_log_ppl:np.float64(0.7462819424690679) - rollout_corr/log_ppl_diff:np.float64(0.0006052012322470546) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013254067162051797) - rollout_corr/log_ppl_diff_max:np.float64(0.006841838359832764) - rollout_corr/log_ppl_diff_min:np.float64(-0.004772186279296875) - rollout_corr/ppl_ratio:np.float64(1.0006067838985473) - rollout_corr/chi2_token:np.float64(0.0014893666375428438) - rollout_corr/chi2_seq:np.float64(1.7015499353874475) - actor/pg_loss:np.float64(-4.240649286657572e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019128616168018198) - actor/ppo_kl:np.float64(-0.00013246856422000075) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3110559768974781) - perf/mfu/actor:np.float64(0.11562384315159505) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.0516357421875) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004874637350440025 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004874637350440025 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:666.81640625 - response_length/max:1727.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:666.81640625 - response_length_non_aborted/max:1727.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011752359569072723 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0456913970410824) - timing_s/agent_loop/generate_sequences/max:np.float64(14.015774877741933) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.352618356359017) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.015774877741933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1727 - timing_s/gen:21.136593686416745 - timing_s/reward:0.07471263594925404 - timing_s/old_log_prob:3.2577999029308558 - timing_s/adv:0.11815616860985756 - timing_s/update_actor:13.682143926620483 - timing_s/step:38.35702735558152 - timing_s/stop_profile:0.00011106766760349274 - timing_per_token_ms/adv:0.0004949384809129078 - timing_per_token_ms/gen:0.12381941762934152 - timing_per_token_ms/update_actor:0.05731245021183218 - perf/total_num_tokens:238729 - perf/time_per_step:38.35702735558152 - perf/throughput:777.9832551507063 (TaskRunner pid=2224264) Training Progress: 48%|████▊ | 48/100 [44:34<41:11, 47.54s/it] (TaskRunner pid=2224264) step:49 - global_seqlen/min:27565 - global_seqlen/max:33967 - global_seqlen/minmax_diff:6402 - global_seqlen/balanced_min:31660 - global_seqlen/balanced_max:31677 - global_seqlen/mean:31668.625 - actor/entropy:0.7485659122467041 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8766915202140808 - training/rollout_probs_diff_mean:0.004531657788902521 - training/rollout_probs_diff_std:0.008790009655058384 - training/rollout_actor_probs_pearson_corr:0.9996523857116699 - rollout_corr/rollout_is_mean:1.0000706911087036 - rollout_corr/rollout_is_ratio_fraction_high:5.495320692716632e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.495320692716632e-06 - rollout_corr/rollout_is_max:8.354974746704102 - rollout_corr/rollout_is_min:0.0006339381216093898 - rollout_corr/rollout_is_std:0.03499140590429306 - rollout_corr/rollout_is_eff_sample_size:0.99877733654154 - rollout_corr/rollout_is_seq_mean:1.0000840425491333 - rollout_corr/rollout_is_seq_std:0.0015765647403895855 - rollout_corr/rollout_is_seq_max:1.0120781660079956 - rollout_corr/rollout_is_seq_min:0.9965959787368774 - rollout_corr/rollout_is_seq_max_deviation:0.012078166007995605 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.135033040307462) - rollout_corr/training_log_ppl:np.float64(0.7435363659169525) - rollout_corr/kl:np.float64(0.000531884074939093) - rollout_corr/k3_kl:np.float64(0.0007136939013889787) - rollout_corr/rollout_ppl:np.float64(2.1338718575425446) - rollout_corr/rollout_log_ppl:np.float64(0.7430044782813638) - rollout_corr/log_ppl_diff:np.float64(0.0005318876355886459) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775929015129805) - rollout_corr/log_ppl_diff_max:np.float64(0.011757910251617432) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038299560546875) - rollout_corr/ppl_ratio:np.float64(1.0005332818254828) - rollout_corr/chi2_token:np.float64(0.00199099606834352) - rollout_corr/chi2_seq:np.float64(1.2359290518797934) - actor/pg_loss:np.float64(-0.00026974163483828306) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014913909058122954) - actor/ppo_kl:np.float64(-6.670435310951461e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24813814461231232) - perf/mfu/actor:np.float64(0.12164250497550971) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.1574478149414) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006872585508972406 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006872585508972406 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:710.83203125 - response_length/max:1348.0 - response_length/min:200.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:710.83203125 - response_length_non_aborted/max:1348.0 - response_length_non_aborted/min:200.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015538372099399567 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5202769823372364) - timing_s/agent_loop/generate_sequences/max:np.float64(13.002393383532763) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.453353088778385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.002393383532763) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:1348 - timing_s/gen:20.19887711107731 - timing_s/reward:0.077494902536273 - timing_s/old_log_prob:3.275420231744647 - timing_s/adv:0.1729813516139984 - timing_s/update_actor:13.726705972105265 - timing_s/step:37.536205399781466 - timing_s/stop_profile:0.00011497922241687775 - timing_per_token_ms/adv:0.0006827789003074747 - timing_per_token_ms/gen:0.1109993081999929 - timing_per_token_ms/update_actor:0.05418101501132929 - perf/total_num_tokens:253349 - perf/time_per_step:37.536205399781466 - perf/throughput:843.6821107171471 (TaskRunner pid=2224264) Training Progress: 49%|████▉ | 49/100 [45:11<37:52, 44.56s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product molecule: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) This is a **nitrogen-containing heterocyclic compound** (likely a pyrrole or similar), with a substituent on the nitrogen: (TaskRunner pid=2224264) - A **benzene ring** with a **chloro group** on one carbon (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F) on another carbon (TaskRunner pid=2224264) - A **nitrile group** (the N in the ring is bonded to a carbon chain) (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let's break it down: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Product structure: (TaskRunner pid=2224264) - The core is a **pyrrole-like ring** (N in ring) (TaskRunner pid=2224264) - Attached to the N is a **benzene ring** with: (TaskRunner pid=2224264) - A **chloro group** (Cl) on one carbon (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F) on another carbon (TaskRunner pid=2224264) - The ring also has a **nitrile group** (C≡N) (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are to determine which of the four options is the **only correct reactant** in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option A:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **nitrile group** (C≡N) (TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and C(F)(F)F (TaskRunner pid=2224264) - The structure matches the **product** closely (TaskRunner pid=2224264) - The **nitrogen ring** is connected to the same substituents (TaskRunner pid=2224264) - Seems to be a **correct structure** for a possible reactant (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option B:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **Cl group** on the ring (TaskRunner pid=2224264) - The ring has **nitrogen** and **chlorine** (TaskRunner pid=2224264) - However, the **trifluoromethyl group** is missing (TaskRunner pid=2224264) - Structure does **not match** the product (TaskRunner pid=2224264) - Likely incorrect (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option C:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and C(F)(F)F (TaskRunner pid=2224264) - Contains **nitrogen and oxygen** in the ring (TaskRunner pid=2224264) - However, the **nitrile group** is not present (TaskRunner pid=2224264) - The **structure is not the same** as the product (TaskRunner pid=2224264) - Likely incorrect (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option D:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **trifluoromethyl group** (TaskRunner pid=2224264) - Contains **double bonds and rings** (TaskRunner pid=2224264) - But **no benzene ring with Cl** (TaskRunner pid=2224264) - No **nitrogen ring** with the correct substituents (TaskRunner pid=2224264) - Does not match the product (TaskRunner pid=2224264) - Incorrect (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Conclusion:** (TaskRunner pid=2224264) Only **Option A** contains all the necessary **functional groups and substituents** to be the correct reactant in the synthesis of the given product. It matches the **structure and substituents** of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2224264) step:50 - global_seqlen/min:25774 - global_seqlen/max:40010 - global_seqlen/minmax_diff:14236 - global_seqlen/balanced_min:30199 - global_seqlen/balanced_max:30236 - global_seqlen/mean:30221.125 - actor/entropy:0.6831239461898804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18094828724861145 - training/rollout_probs_diff_mean:0.00435651233419776 - training/rollout_probs_diff_std:0.008434131741523743 - training/rollout_actor_probs_pearson_corr:0.9996002316474915 - rollout_corr/rollout_is_mean:0.9999532699584961 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.877063358639134e-06 - rollout_corr/rollout_is_max:1.8264771699905396 - rollout_corr/rollout_is_min:0.4736720621585846 - rollout_corr/rollout_is_std:0.03403113782405853 - rollout_corr/rollout_is_eff_sample_size:0.998842864637548 - rollout_corr/rollout_is_seq_mean:0.9999036192893982 - rollout_corr/rollout_is_seq_std:0.0016001641051843762 - rollout_corr/rollout_is_seq_max:1.0056898593902588 - rollout_corr/rollout_is_seq_min:0.9943985939025879 - rollout_corr/rollout_is_seq_max_deviation:0.005689859390258789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0370463258586824) - rollout_corr/training_log_ppl:np.float64(0.6895827453117818) - rollout_corr/kl:np.float64(0.0007133520028830276) - rollout_corr/k3_kl:np.float64(0.0006415439404463541) - rollout_corr/rollout_ppl:np.float64(2.035565957427025) - rollout_corr/rollout_log_ppl:np.float64(0.6888693935470656) - rollout_corr/log_ppl_diff:np.float64(0.000713351764716208) - rollout_corr/log_ppl_abs_diff:np.float64(0.001375065534375608) - rollout_corr/log_ppl_diff_max:np.float64(0.010087192058563232) - rollout_corr/log_ppl_diff_min:np.float64(-0.004953622817993164) - rollout_corr/ppl_ratio:np.float64(1.0007150897290558) - rollout_corr/chi2_token:np.float64(0.0011441940441727638) - rollout_corr/chi2_seq:np.float64(1.4973946234676987) - actor/pg_loss:np.float64(0.00014200282748788595) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030153528336995805) - actor/ppo_kl:np.float64(1.1964239081230232e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3060915023088455) - perf/mfu/actor:np.float64(0.11810264114100524) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.11758041381836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6223214285714286) - val-aux/sciknoweval/reward/std@16:np.float64(0.2647595134884485) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7291095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19877560636838412) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5156142857142857) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24700525701136392) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6215809523809525) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2629386719576279) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8044857142857144) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12973275443729954) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4167428571428572) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.20684059335831737) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6544095238095238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20967583799172174) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8527809523809524) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07656020696044112) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3329095238095238) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.16042430484607695) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6712285714285714) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15275032646284742) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8799809523809523) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.041193057541303105) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.2648619047619048) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.10865600566624517) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6785476190476191) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11458915286528892) - val-aux/sciknoweval/score/mean@16:np.float64(0.6223214285714286) - val-aux/sciknoweval/score/std@16:np.float64(0.2647595134884485) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7291095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19877560636838412) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5156142857142857) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24700525701136392) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6215809523809525) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2629386719576279) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8044857142857144) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12973275443729954) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4167428571428572) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.20684059335831737) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6544095238095238) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20967583799172174) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8527809523809524) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07656020696044112) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3329095238095238) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.16042430484607695) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6712285714285714) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15275032646284742) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8799809523809523) - val-aux/sciknoweval/score/best@16/std:np.float64(0.041193057541303105) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.2648619047619048) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.10865600566624517) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6785476190476191) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11458915286528892) - val-core/sciknoweval/acc/mean@16:np.float64(0.6223214285714286) - val-aux/sciknoweval/acc/std@16:np.float64(0.2647595134884485) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7291095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19877560636838412) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5156142857142857) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24700525701136392) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6215809523809525) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2629386719576279) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8044857142857144) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12973275443729954) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4167428571428572) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.20684059335831737) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6544095238095238) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20967583799172174) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8527809523809524) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07656020696044112) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3329095238095238) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.16042430484607695) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6712285714285714) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15275032646284742) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8799809523809523) - val-core/sciknoweval/acc/best@16/std:np.float64(0.041193057541303105) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.2648619047619048) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.10865600566624517) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6785476190476191) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11458915286528892) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.986904761904762) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03936087691755317) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.998252380952381) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013004708934596106) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9755047619047619) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0512654983800644) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9868380952380952) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.039356229004372134) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999285714285715) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0010041883502657082) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9567476190476191) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06208854001798514) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9947333333333334) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.024839640974003308) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9297809523809525) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06686724116120492) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9986714285714285) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011269482933237476) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8981190476190476) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05941598517810314) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998619047619047) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0028135293122891663) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002585907932370901 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002585907932370901 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:664.66015625 - response_length/max:1839.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:664.66015625 - response_length_non_aborted/max:1839.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018392875790596008 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0269889794290066) - timing_s/agent_loop/generate_sequences/max:np.float64(14.812706418335438) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.506310426571872) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.812706418335438) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:165 - timing_s/agent_loop/slowest/response_length:1763 - timing_s/gen:21.724913926795125 - timing_s/reward:0.07297280617058277 - timing_s/old_log_prob:3.25269278883934 - timing_s/adv:0.11958634853363037 - timing_s/update_actor:13.451122742146254 - timing_s/step:38.707119492813945 - timing_s/testing:191.03151337988675 - timing_s/save_checkpoint:12.945183955132961 - timing_s/stop_profile:0.00014170818030834198 - timing_per_token_ms/adv:0.000494630612417764 - timing_per_token_ms/gen:0.1276787005036357 - timing_per_token_ms/update_actor:0.05563625916534483 - perf/total_num_tokens:241769 - perf/time_per_step:38.707119492813945 - perf/throughput:780.7639885373184 (TaskRunner pid=2224264) Training Progress: 50%|█████ | 50/100 [49:14<1:26:40, 104.01s/it] (TaskRunner pid=2224264) step:51 - global_seqlen/min:30736 - global_seqlen/max:37371 - global_seqlen/minmax_diff:6635 - global_seqlen/balanced_min:34292 - global_seqlen/balanced_max:34302 - global_seqlen/mean:34297.75 - actor/entropy:0.762580394744873 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294635772705078 - training/rollout_probs_diff_mean:0.004454846493899822 - training/rollout_probs_diff_std:0.008279886096715927 - training/rollout_actor_probs_pearson_corr:0.9996731281280518 - rollout_corr/rollout_is_mean:1.0000529289245605 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.811339259147644 - rollout_corr/rollout_is_min:0.5012558698654175 - rollout_corr/rollout_is_std:0.03488221764564514 - rollout_corr/rollout_is_eff_sample_size:0.9987848284293472 - rollout_corr/rollout_is_seq_mean:1.0000650882720947 - rollout_corr/rollout_is_seq_std:0.0014846812700852752 - rollout_corr/rollout_is_seq_max:1.0048807859420776 - rollout_corr/rollout_is_seq_min:0.9954461455345154 - rollout_corr/rollout_is_seq_max_deviation:0.004880785942077637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.208350053522736) - rollout_corr/training_log_ppl:np.float64(0.7750773588195443) - rollout_corr/kl:np.float64(0.0005686312180488429) - rollout_corr/k3_kl:np.float64(0.0006700464483628821) - rollout_corr/rollout_ppl:np.float64(2.207024629227817) - rollout_corr/rollout_log_ppl:np.float64(0.7745087213115767) - rollout_corr/log_ppl_diff:np.float64(0.0005686375079676509) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012317007640376687) - rollout_corr/log_ppl_diff_max:np.float64(0.006791532039642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.006045103073120117) - rollout_corr/ppl_ratio:np.float64(1.000570009695366) - rollout_corr/chi2_token:np.float64(0.0015524374321103096) - rollout_corr/chi2_seq:np.float64(2.341128587955609) - actor/pg_loss:np.float64(-0.00025634258054196835) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016806365397314948) - actor/ppo_kl:np.float64(7.1047582039796e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24387626722455025) - perf/mfu/actor:np.float64(0.13125870218702868) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.71975231170654) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033099849242717028 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033099849242717028 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.2734375 - response_length/max:1708.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.2734375 - response_length_non_aborted/max:1708.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.316764771938324e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.95208290964365) - timing_s/agent_loop/generate_sequences/max:np.float64(15.170884735882282) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.576293181155052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.170884735882282) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:1708 - timing_s/gen:21.975822178646922 - timing_s/reward:0.07666116580367088 - timing_s/old_log_prob:3.373812895268202 - timing_s/adv:0.12022814154624939 - timing_s/update_actor:13.922087775543332 - timing_s/step:39.56273279339075 - timing_s/stop_profile:0.00014520063996315002 - timing_per_token_ms/adv:0.0004381779473371044 - timing_per_token_ms/gen:0.11305947389386915 - timing_per_token_ms/update_actor:0.05073979989774596 - perf/total_num_tokens:274382 - perf/time_per_step:39.56273279339075 - perf/throughput:866.920649266415 (TaskRunner pid=2224264) Training Progress: 51%|█████ | 51/100 [49:54<1:09:09, 84.69s/it] (TaskRunner pid=2224264) step:52 - global_seqlen/min:25163 - global_seqlen/max:39354 - global_seqlen/minmax_diff:14191 - global_seqlen/balanced_min:31304 - global_seqlen/balanced_max:31326 - global_seqlen/mean:31315.625 - actor/entropy:0.7275306582450867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846359670162201 - training/rollout_probs_diff_mean:0.0044688270427286625 - training/rollout_probs_diff_std:0.008479300886392593 - training/rollout_actor_probs_pearson_corr:0.9996131658554077 - rollout_corr/rollout_is_mean:0.9999827146530151 - rollout_corr/rollout_is_ratio_fraction_high:5.457889528770465e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.091577905754093e-05 - rollout_corr/rollout_is_max:2.513453245162964 - rollout_corr/rollout_is_min:0.2490382194519043 - rollout_corr/rollout_is_std:0.03466796875 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:0.9999496936798096 - rollout_corr/rollout_is_seq_std:0.0013713843654841185 - rollout_corr/rollout_is_seq_max:1.0041810274124146 - rollout_corr/rollout_is_seq_min:0.9949362277984619 - rollout_corr/rollout_is_seq_max_deviation:0.005063772201538086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0971961123868823) - rollout_corr/training_log_ppl:np.float64(0.7249851016094908) - rollout_corr/kl:np.float64(0.000624122314107467) - rollout_corr/k3_kl:np.float64(0.0006732126169026742) - rollout_corr/rollout_ppl:np.float64(2.095865353010595) - rollout_corr/rollout_log_ppl:np.float64(0.7243609796278179) - rollout_corr/log_ppl_diff:np.float64(0.0006241219816729426) - rollout_corr/log_ppl_abs_diff:np.float64(0.001236821641214192) - rollout_corr/log_ppl_diff_max:np.float64(0.00571441650390625) - rollout_corr/log_ppl_diff_min:np.float64(-0.005221307277679443) - rollout_corr/ppl_ratio:np.float64(1.0006253765895963) - rollout_corr/chi2_token:np.float64(0.0014572420623153448) - rollout_corr/chi2_seq:np.float64(2.134200472617522) - actor/pg_loss:np.float64(0.00022022065240889788) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021542626041082258) - actor/ppo_kl:np.float64(-4.695473637639225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25628167018294334) - perf/mfu/actor:np.float64(0.1214559238638076) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.17374038696289) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011097254231572151 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011097254231572151 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:715.70703125 - response_length/max:1611.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:715.70703125 - response_length_non_aborted/max:1611.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.15229320526123e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3523097801953554) - timing_s/agent_loop/generate_sequences/max:np.float64(14.437729807570577) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.303848396521062) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.437729807570577) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1611 - timing_s/gen:21.366347413510084 - timing_s/reward:0.07527691684663296 - timing_s/old_log_prob:3.179901920258999 - timing_s/adv:0.12300918810069561 - timing_s/update_actor:13.513995960354805 - timing_s/step:38.342299580574036 - timing_s/stop_profile:0.00011920370161533356 - timing_per_token_ms/adv:0.0004910056405576115 - timing_per_token_ms/gen:0.1166151664575026 - timing_per_token_ms/update_actor:0.053942704162677595 - perf/total_num_tokens:250525 - perf/time_per_step:38.342299580574036 - perf/throughput:816.7383110184119 (TaskRunner pid=2224264) Training Progress: 52%|█████▏ | 52/100 [50:32<56:38, 70.80s/it] (TaskRunner pid=2224264) step:53 - global_seqlen/min:29023 - global_seqlen/max:38170 - global_seqlen/minmax_diff:9147 - global_seqlen/balanced_min:32089 - global_seqlen/balanced_max:32125 - global_seqlen/mean:32112.5 - actor/entropy:0.7673122882843018 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2821834683418274 - training/rollout_probs_diff_mean:0.004418913275003433 - training/rollout_probs_diff_std:0.008327017538249493 - training/rollout_actor_probs_pearson_corr:0.9996277093887329 - rollout_corr/rollout_is_mean:1.0000674724578857 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6528507471084595 - rollout_corr/rollout_is_min:0.6141930818557739 - rollout_corr/rollout_is_std:0.035042472183704376 - rollout_corr/rollout_is_eff_sample_size:0.9987737690154207 - rollout_corr/rollout_is_seq_mean:1.0000889301300049 - rollout_corr/rollout_is_seq_std:0.0013838375452905893 - rollout_corr/rollout_is_seq_max:1.005210518836975 - rollout_corr/rollout_is_seq_min:0.9962599873542786 - rollout_corr/rollout_is_seq_max_deviation:0.005210518836975098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2401722352951765) - rollout_corr/training_log_ppl:np.float64(0.7886160378111526) - rollout_corr/kl:np.float64(0.0006792797128305494) - rollout_corr/k3_kl:np.float64(0.0006657197256458858) - rollout_corr/rollout_ppl:np.float64(2.238586582709104) - rollout_corr/rollout_log_ppl:np.float64(0.7879367553978227) - rollout_corr/log_ppl_diff:np.float64(0.0006792824133299291) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012466523912735283) - rollout_corr/log_ppl_diff_max:np.float64(0.0049315690994262695) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511535167694092) - rollout_corr/ppl_ratio:np.float64(1.000680540688336) - rollout_corr/chi2_token:np.float64(0.00131767219863832) - rollout_corr/chi2_seq:np.float64(0.9375523473136127) - actor/pg_loss:np.float64(-5.056627560406923e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015928275297483196) - actor/ppo_kl:np.float64(0.00014284484250026708) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2610822059214115) - perf/mfu/actor:np.float64(0.12400191776249886) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.9232177734375) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009051043540239334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009051043540239334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:707.953125 - response_length/max:1922.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:707.953125 - response_length_non_aborted/max:1922.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.531714022159576e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6737753115594387) - timing_s/agent_loop/generate_sequences/max:np.float64(16.309401037171483) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.246375748174614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.309401037171483) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:1922 - timing_s/gen:22.928913297131658 - timing_s/reward:0.07421708479523659 - timing_s/old_log_prob:3.3302101399749517 - timing_s/adv:0.1337327789515257 - timing_s/update_actor:13.578472383320332 - timing_s/step:40.13115260377526 - timing_s/stop_profile:0.0001122206449508667 - timing_per_token_ms/adv:0.000520563561508469 - timing_per_token_ms/gen:0.12651412135078935 - timing_per_token_ms/update_actor:0.0528550890748164 - perf/total_num_tokens:256900 - perf/time_per_step:40.13115260377526 - perf/throughput:800.1888287898084 (TaskRunner pid=2224264) Training Progress: 53%|█████▎ | 53/100 [51:12<48:15, 61.61s/it] (TaskRunner pid=2224264) step:54 - global_seqlen/min:24046 - global_seqlen/max:37658 - global_seqlen/minmax_diff:13612 - global_seqlen/balanced_min:31667 - global_seqlen/balanced_max:31732 - global_seqlen/mean:31711.0 - actor/entropy:0.7221550345420837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564734995365143 - training/rollout_probs_diff_mean:0.0042567262426018715 - training/rollout_probs_diff_std:0.008278296329081059 - training/rollout_actor_probs_pearson_corr:0.9996740221977234 - rollout_corr/rollout_is_mean:0.9999973773956299 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.423934908321826e-06 - rollout_corr/rollout_is_max:1.8484896421432495 - rollout_corr/rollout_is_min:0.4127388894557953 - rollout_corr/rollout_is_std:0.03383616358041763 - rollout_corr/rollout_is_eff_sample_size:0.9988563043114053 - rollout_corr/rollout_is_seq_mean:0.9999293088912964 - rollout_corr/rollout_is_seq_std:0.0013600483071058989 - rollout_corr/rollout_is_seq_max:1.003616213798523 - rollout_corr/rollout_is_seq_min:0.9944389462471008 - rollout_corr/rollout_is_seq_max_deviation:0.00556105375289917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1455902410671115) - rollout_corr/training_log_ppl:np.float64(0.7426831858465448) - rollout_corr/kl:np.float64(0.0007347915104665148) - rollout_corr/k3_kl:np.float64(0.0006544318975443275) - rollout_corr/rollout_ppl:np.float64(2.14404917974025) - rollout_corr/rollout_log_ppl:np.float64(0.7419483949197456) - rollout_corr/log_ppl_diff:np.float64(0.0007347909267991781) - rollout_corr/log_ppl_abs_diff:np.float64(0.00128568010404706) - rollout_corr/log_ppl_diff_max:np.float64(0.007453739643096924) - rollout_corr/log_ppl_diff_min:np.float64(-0.0039052367210388184) - rollout_corr/ppl_ratio:np.float64(1.000736182089895) - rollout_corr/chi2_token:np.float64(0.0011647003702819347) - rollout_corr/chi2_seq:np.float64(1.3221846474334598) - actor/pg_loss:np.float64(-0.0001485362881794572) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030488177571896813) - actor/ppo_kl:np.float64(6.0147084293804554e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3080219365656376) - perf/mfu/actor:np.float64(0.12202071329811175) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.63494873046875) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016614189371466637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.016614189371466637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:720.1875 - response_length/max:2286.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:720.1875 - response_length_non_aborted/max:2286.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.760074317455292e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9534575715661049) - timing_s/agent_loop/generate_sequences/max:np.float64(17.89197232015431) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.935718771463144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.89197232015431) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:2286 - timing_s/gen:25.277695801109076 - timing_s/reward:0.08129468001425266 - timing_s/old_log_prob:3.2715712431818247 - timing_s/adv:0.1201015543192625 - timing_s/update_actor:13.721273077651858 - timing_s/step:42.558674819767475 - timing_s/stop_profile:0.00013141706585884094 - timing_per_token_ms/adv:0.0004734222916309108 - timing_per_token_ms/gen:0.13710457238300072 - timing_per_token_ms/update_actor:0.05408719796620991 - perf/total_num_tokens:253688 - perf/time_per_step:42.558674819767475 - perf/throughput:745.1124860981575 (TaskRunner pid=2224264) Training Progress: 54%|█████▍ | 54/100 [51:55<42:51, 55.91s/it] (TaskRunner pid=2224264) step:55 - global_seqlen/min:29461 - global_seqlen/max:36701 - global_seqlen/minmax_diff:7240 - global_seqlen/balanced_min:33720 - global_seqlen/balanced_max:34509 - global_seqlen/mean:33827.375 - actor/entropy:0.8024973273277283 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23547464609146118 - training/rollout_probs_diff_mean:0.0043955049477517605 - training/rollout_probs_diff_std:0.008106191642582417 - training/rollout_actor_probs_pearson_corr:0.999698281288147 - rollout_corr/rollout_is_mean:1.0000178813934326 - rollout_corr/rollout_is_ratio_fraction_high:5.190302545088343e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0640006065368652 - rollout_corr/rollout_is_min:0.5909958481788635 - rollout_corr/rollout_is_std:0.03487367555499077 - rollout_corr/rollout_is_eff_sample_size:0.9987854230284318 - rollout_corr/rollout_is_seq_mean:0.999980092048645 - rollout_corr/rollout_is_seq_std:0.0013782148016616702 - rollout_corr/rollout_is_seq_max:1.0047253370285034 - rollout_corr/rollout_is_seq_min:0.9960294961929321 - rollout_corr/rollout_is_seq_max_deviation:0.004725337028503418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.3131857896223664) - rollout_corr/training_log_ppl:np.float64(0.8200820172205567) - rollout_corr/kl:np.float64(0.0007159894126873567) - rollout_corr/k3_kl:np.float64(0.0006568527235231159) - rollout_corr/rollout_ppl:np.float64(2.311495006084442) - rollout_corr/rollout_log_ppl:np.float64(0.8193660224787891) - rollout_corr/log_ppl_diff:np.float64(0.0007159947417676449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012078655418008566) - rollout_corr/log_ppl_diff_max:np.float64(0.004928708076477051) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037838220596313477) - rollout_corr/ppl_ratio:np.float64(1.0007172094192356) - rollout_corr/chi2_token:np.float64(0.0011943182907998562) - rollout_corr/chi2_seq:np.float64(2.1422022855840623) - actor/pg_loss:np.float64(6.403832230716944e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014861452268633002) - actor/ppo_kl:np.float64(7.01219851411139e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2634842284023762) - perf/mfu/actor:np.float64(0.12720097629374041) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.60932922363281) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000431443884735927 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000431443884735927 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:752.60546875 - response_length/max:2800.0 - response_length/min:187.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:752.60546875 - response_length_non_aborted/max:2800.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.310990571975708e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7985909432172775) - timing_s/agent_loop/generate_sequences/max:np.float64(20.643996415659785) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.059685343854653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.643996415659785) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:2800 - timing_s/gen:27.551894007250667 - timing_s/reward:0.07855446077883244 - timing_s/old_log_prob:3.331155611202121 - timing_s/adv:0.1180018950253725 - timing_s/update_actor:14.12481613457203 - timing_s/step:45.289689207449555 - timing_s/stop_profile:3.606826066970825e-05 - timing_per_token_ms/adv:0.0004360443835258149 - timing_per_token_ms/gen:0.14300266266278433 - timing_per_token_ms/update_actor:0.05219447316918631 - perf/total_num_tokens:270619 - perf/time_per_step:45.289689207449555 - perf/throughput:746.9111754124345 (TaskRunner pid=2224264) Training Progress: 55%|█████▌ | 55/100 [52:40<39:32, 52.73s/it] (TaskRunner pid=2224264) step:56 - global_seqlen/min:25792 - global_seqlen/max:39171 - global_seqlen/minmax_diff:13379 - global_seqlen/balanced_min:31985 - global_seqlen/balanced_max:31996 - global_seqlen/mean:31991.875 - actor/entropy:0.8143277764320374 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2929895520210266 - training/rollout_probs_diff_mean:0.004465053323656321 - training/rollout_probs_diff_std:0.008296534419059753 - training/rollout_actor_probs_pearson_corr:0.9996805787086487 - rollout_corr/rollout_is_mean:0.9999870657920837 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5952778994687833e-05 - rollout_corr/rollout_is_max:1.629705786705017 - rollout_corr/rollout_is_min:0.2926250994205475 - rollout_corr/rollout_is_std:0.034940268844366074 - rollout_corr/rollout_is_eff_sample_size:0.9987806662555786 - rollout_corr/rollout_is_seq_mean:1.0000478029251099 - rollout_corr/rollout_is_seq_std:0.0013941883808001876 - rollout_corr/rollout_is_seq_max:1.0048588514328003 - rollout_corr/rollout_is_seq_min:0.9960804581642151 - rollout_corr/rollout_is_seq_max_deviation:0.004858851432800293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.281407651025802) - rollout_corr/training_log_ppl:np.float64(0.804402201261837) - rollout_corr/kl:np.float64(0.0005219981214015235) - rollout_corr/k3_kl:np.float64(0.0006556217045954327) - rollout_corr/rollout_ppl:np.float64(2.2802593959495425) - rollout_corr/rollout_log_ppl:np.float64(0.803880195133388) - rollout_corr/log_ppl_diff:np.float64(0.000522006128448993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012433496885932982) - rollout_corr/log_ppl_diff_max:np.float64(0.0066702961921691895) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035393238067626953) - rollout_corr/ppl_ratio:np.float64(1.0005232789553702) - rollout_corr/chi2_token:np.float64(0.0015740657690912485) - rollout_corr/chi2_seq:np.float64(5.73605796508491) - actor/pg_loss:np.float64(-4.861317574977875e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018069532825393253) - actor/ppo_kl:np.float64(-5.679699522431747e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2360125184059143) - perf/mfu/actor:np.float64(0.12372654611791109) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.54317474365234) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002341070445254445 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002341070445254445 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:734.58984375 - response_length/max:1632.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:734.58984375 - response_length_non_aborted/max:1632.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9824674129486084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.308333108201623) - timing_s/agent_loop/generate_sequences/max:np.float64(14.604689434170723) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.345092790725175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.604689434170723) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:1632 - timing_s/gen:21.219629768282175 - timing_s/reward:0.07743704505264759 - timing_s/old_log_prob:3.259379044175148 - timing_s/adv:0.11571713909506798 - timing_s/update_actor:13.64204434491694 - timing_s/step:38.399653954431415 - timing_s/stop_profile:0.00011725537478923798 - timing_per_token_ms/adv:0.0004521348744605778 - timing_per_token_ms/gen:0.11283736017804459 - timing_per_token_ms/update_actor:0.05330276962868283 - perf/total_num_tokens:255935 - perf/time_per_step:38.399653954431415 - perf/throughput:833.1292526220294 (TaskRunner pid=2224264) Training Progress: 56%|█████▌ | 56/100 [53:19<35:31, 48.45s/it] (TaskRunner pid=2224264) step:57 - global_seqlen/min:27687 - global_seqlen/max:34593 - global_seqlen/minmax_diff:6906 - global_seqlen/balanced_min:31298 - global_seqlen/balanced_max:31311 - global_seqlen/mean:31305.125 - actor/entropy:0.7938913106918335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.680421769618988 - training/rollout_probs_diff_mean:0.004562178626656532 - training/rollout_probs_diff_std:0.008708939887583256 - training/rollout_actor_probs_pearson_corr:0.9995999932289124 - rollout_corr/rollout_is_mean:0.9999580979347229 - rollout_corr/rollout_is_ratio_fraction_high:5.648919795930851e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6946758478297852e-05 - rollout_corr/rollout_is_max:2.2787086963653564 - rollout_corr/rollout_is_min:0.24652595818042755 - rollout_corr/rollout_is_std:0.035597678273916245 - rollout_corr/rollout_is_eff_sample_size:0.9987344090023405 - rollout_corr/rollout_is_seq_mean:0.9999576807022095 - rollout_corr/rollout_is_seq_std:0.001491630100645125 - rollout_corr/rollout_is_seq_max:1.0051113367080688 - rollout_corr/rollout_is_seq_min:0.9956648349761963 - rollout_corr/rollout_is_seq_max_deviation:0.005111336708068848 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.283419963438064) - rollout_corr/training_log_ppl:np.float64(0.8058918532915413) - rollout_corr/kl:np.float64(0.0006797224436070337) - rollout_corr/k3_kl:np.float64(0.0007055759461991329) - rollout_corr/rollout_ppl:np.float64(2.281822293996811) - rollout_corr/rollout_log_ppl:np.float64(0.80521212273743) - rollout_corr/log_ppl_diff:np.float64(0.0006797305541113019) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013461158378049731) - rollout_corr/log_ppl_diff_max:np.float64(0.0058408379554748535) - rollout_corr/log_ppl_diff_min:np.float64(-0.004617094993591309) - rollout_corr/ppl_ratio:np.float64(1.0006813008803874) - rollout_corr/chi2_token:np.float64(0.0014507242012768984) - rollout_corr/chi2_seq:np.float64(1.6866442584432662) - actor/pg_loss:np.float64(1.2409058399498463e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002567762589933409) - actor/ppo_kl:np.float64(-1.4622721524304438e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2836088426411152) - perf/mfu/actor:np.float64(0.12280457724250629) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.51215362548828) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012533540138974786 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012533540138974786 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:691.50390625 - response_length/max:1421.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:691.50390625 - response_length_non_aborted/max:1421.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012706033885478973 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5593209099024534) - timing_s/agent_loop/generate_sequences/max:np.float64(12.716230429708958) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.830025155824842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.716230429708958) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1334 - timing_s/gen:19.76116858050227 - timing_s/reward:0.08693374507129192 - timing_s/old_log_prob:3.292963979765773 - timing_s/adv:0.1231906358152628 - timing_s/update_actor:13.490689365193248 - timing_s/step:36.84112096019089 - timing_s/stop_profile:0.00011323578655719757 - timing_per_token_ms/adv:0.0004918948407619471 - timing_per_token_ms/gen:0.11162925338512793 - timing_per_token_ms/update_actor:0.05386773477662702 - perf/total_num_tokens:250441 - perf/time_per_step:36.84112096019089 - perf/throughput:849.7332378628523 (TaskRunner pid=2224264) Training Progress: 57%|█████▋ | 57/100 [53:56<32:14, 44.98s/it] (TaskRunner pid=2224264) step:58 - global_seqlen/min:26170 - global_seqlen/max:34255 - global_seqlen/minmax_diff:8085 - global_seqlen/balanced_min:30104 - global_seqlen/balanced_max:30420 - global_seqlen/mean:30161.5 - actor/entropy:0.7656847238540649 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16260474920272827 - training/rollout_probs_diff_mean:0.00442152377218008 - training/rollout_probs_diff_std:0.00836502481251955 - training/rollout_actor_probs_pearson_corr:0.9997066259384155 - rollout_corr/rollout_is_mean:0.9998526573181152 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.867856089025736e-06 - rollout_corr/rollout_is_max:1.6352856159210205 - rollout_corr/rollout_is_min:0.49183452129364014 - rollout_corr/rollout_is_std:0.034748680889606476 - rollout_corr/rollout_is_eff_sample_size:0.9987937474899505 - rollout_corr/rollout_is_seq_mean:0.9999055862426758 - rollout_corr/rollout_is_seq_std:0.0014970327029004693 - rollout_corr/rollout_is_seq_max:1.0064585208892822 - rollout_corr/rollout_is_seq_min:0.9960758090019226 - rollout_corr/rollout_is_seq_max_deviation:0.0064585208892822266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.18733112514019) - rollout_corr/training_log_ppl:np.float64(0.7628000770928338) - rollout_corr/kl:np.float64(0.0007255446162326429) - rollout_corr/k3_kl:np.float64(0.0006535338600883733) - rollout_corr/rollout_ppl:np.float64(2.185690295882523) - rollout_corr/rollout_log_ppl:np.float64(0.7620745294261724) - rollout_corr/log_ppl_diff:np.float64(0.0007255476666614413) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013074638554826379) - rollout_corr/log_ppl_diff_max:np.float64(0.005662679672241211) - rollout_corr/log_ppl_diff_min:np.float64(-0.00701594352722168) - rollout_corr/ppl_ratio:np.float64(1.000726986443624) - rollout_corr/chi2_token:np.float64(0.0011695893481373787) - rollout_corr/chi2_seq:np.float64(1.832958000479266) - actor/pg_loss:np.float64(1.464621163904667e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012746554830300738) - actor/ppo_kl:np.float64(9.933237482329105e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2319943569600582) - perf/mfu/actor:np.float64(0.11497644796674522) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.57088470458984) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002008273731917143 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002008273731917143 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:665.703125 - response_length/max:2537.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:665.703125 - response_length_non_aborted/max:2537.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001398194581270218 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0110350400209427) - timing_s/agent_loop/generate_sequences/max:np.float64(19.426406839862466) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.346314342365076) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.426406839862466) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:199 - timing_s/agent_loop/slowest/response_length:2537 - timing_s/gen:26.27677822113037 - timing_s/reward:0.07973587512969971 - timing_s/old_log_prob:3.217642493546009 - timing_s/adv:0.12296506203711033 - timing_s/update_actor:13.712465612217784 - timing_s/step:43.495522832497954 - timing_s/stop_profile:0.00010489486157894135 - timing_per_token_ms/adv:0.0005096110191681047 - timing_per_token_ms/gen:0.15418834773577264 - timing_per_token_ms/update_actor:0.056829342092642045 - perf/total_num_tokens:241292 - perf/time_per_step:43.495522832497954 - perf/throughput:693.4391872044505 (TaskRunner pid=2224264) Training Progress: 58%|█████▊ | 58/100 [54:39<31:11, 44.55s/it] (TaskRunner pid=2224264) step:59 - global_seqlen/min:27515 - global_seqlen/max:33648 - global_seqlen/minmax_diff:6133 - global_seqlen/balanced_min:30606 - global_seqlen/balanced_max:31185 - global_seqlen/mean:30683.625 - actor/entropy:0.7695986032485962 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4610300064086914 - training/rollout_probs_diff_mean:0.004400559701025486 - training/rollout_probs_diff_std:0.008376654237508774 - training/rollout_actor_probs_pearson_corr:0.9996274709701538 - rollout_corr/rollout_is_mean:1.0000059604644775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.658765076077543e-06 - rollout_corr/rollout_is_max:1.8190242052078247 - rollout_corr/rollout_is_min:0.3681882917881012 - rollout_corr/rollout_is_std:0.03469890356063843 - rollout_corr/rollout_is_eff_sample_size:0.9987974340815278 - rollout_corr/rollout_is_seq_mean:0.9999397397041321 - rollout_corr/rollout_is_seq_std:0.0016430045943707228 - rollout_corr/rollout_is_seq_max:1.0059504508972168 - rollout_corr/rollout_is_seq_min:0.9930161237716675 - rollout_corr/rollout_is_seq_max_deviation:0.0069838762283325195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2372520426288247) - rollout_corr/training_log_ppl:np.float64(0.7862228258745745) - rollout_corr/kl:np.float64(0.0008618131305837551) - rollout_corr/k3_kl:np.float64(0.0006665370408427407) - rollout_corr/rollout_ppl:np.float64(2.2352824225090444) - rollout_corr/rollout_log_ppl:np.float64(0.7853610130259767) - rollout_corr/log_ppl_diff:np.float64(0.000861812848597765) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014634081162512302) - rollout_corr/log_ppl_diff_max:np.float64(0.008165895938873291) - rollout_corr/log_ppl_diff_min:np.float64(-0.005269348621368408) - rollout_corr/ppl_ratio:np.float64(1.0008637136779726) - rollout_corr/chi2_token:np.float64(0.000947984866797924) - rollout_corr/chi2_seq:np.float64(1.44492331543006) - actor/pg_loss:np.float64(-2.2010761313140392e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016003822292987024) - actor/ppo_kl:np.float64(0.00017181198663673847) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20559313893318176) - perf/mfu/actor:np.float64(0.11867857354144504) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.51495361328125) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009747223230078816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009747223230078816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:690.30078125 - response_length/max:2398.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:690.30078125 - response_length_non_aborted/max:2398.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012865476310253143 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7066904306411743) - timing_s/agent_loop/generate_sequences/max:np.float64(18.255667094141245) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.844594236710691) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.255667094141245) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:2398 - timing_s/gen:25.205973524600267 - timing_s/reward:0.07148762419819832 - timing_s/old_log_prob:3.309969963505864 - timing_s/adv:0.11456507630646229 - timing_s/update_actor:13.433156419545412 - timing_s/step:42.23662491515279 - timing_s/stop_profile:0.00011372379958629608 - timing_per_token_ms/adv:0.00046671912260392266 - timing_per_token_ms/gen:0.1426346844084059 - timing_per_token_ms/update_actor:0.054724451639699564 - perf/total_num_tokens:245469 - perf/time_per_step:42.23662491515279 - perf/throughput:726.4696234994847 (TaskRunner pid=2224264) Training Progress: 59%|█████▉ | 59/100 [55:21<29:58, 43.87s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product molecule: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a condensed chemical formula. Let's interpret it step by step. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 1: Parse the product formula (TaskRunner pid=2224264) The formula: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a shorthand for a benzene ring with substituents. Let's break it down: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - **Nc1ccn(...)**: This is a nitrogen-containing ring. (TaskRunner pid=2224264) - **Cc2ccc(Cl)cc2C(F)(F)F**: This is a side chain attached to the ring via the nitrogen. It contains a benzene ring with a Cl substituent and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2224264) (TaskRunner pid=2224264) So the product is likely a **nitrogen-containing heterocycle (e.g., pyridine or pyrrole)**, with a **benzene ring bearing a Cl and a CF₃ group** attached to it. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 2: Understand what we are looking for (TaskRunner pid=2224264) We are to find the correct **reactant** that would be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) The options are all chemical formulas that may be reactants in a synthesis. We must find the one that matches the **structural features of the product**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 3: Analyze each option (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a **diacyl compound** (two C=O groups), and it includes a **nitrogen ring** with the same side chain as the product: (TaskRunner pid=2224264) - `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2224264) This matches the **side chain** of the product. (TaskRunner pid=2224264) - The `O=C1c2ccccc2C(=O)N1` part suggests a **diketone ring** (like a benzene ring with two ketone groups), which may be a **starting material** for forming the nitrogen ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This **looks like a good candidate** as it contains the same side chain and a structure that could be used to form the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This includes a Cl group on the ring and a nitrogen, but it **does not match the side chain** of the product. It also has a different structure and is not consistent with the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **Cl group**, but the **side chain is different**. It has a **hydroxyl group** that is not in the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **CF₃ group** and a **benzene ring**, but the **side chain is different**. It does not match the structure of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 4: Final determination (TaskRunner pid=2224264) Only **Option A** includes the **same side chain** as the product and includes a **ring structure** that could be used in the synthesis of the nitrogen-containing ring in the product. It is the only one that matches the **structural features** of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2224264) step:60 - global_seqlen/min:28478 - global_seqlen/max:38447 - global_seqlen/minmax_diff:9969 - global_seqlen/balanced_min:32368 - global_seqlen/balanced_max:32372 - global_seqlen/mean:32370.0 - actor/entropy:0.8034039735794067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4281436800956726 - training/rollout_probs_diff_mean:0.004418870899826288 - training/rollout_probs_diff_std:0.008324537426233292 - training/rollout_actor_probs_pearson_corr:0.9996195435523987 - rollout_corr/rollout_is_mean:1.0001076459884644 - rollout_corr/rollout_is_ratio_fraction_high:1.047778732754523e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.238893663772615e-06 - rollout_corr/rollout_is_max:2.1436619758605957 - rollout_corr/rollout_is_min:0.026059480383992195 - rollout_corr/rollout_is_std:0.0351596400141716 - rollout_corr/rollout_is_eff_sample_size:0.9987655638020635 - rollout_corr/rollout_is_seq_mean:1.0000665187835693 - rollout_corr/rollout_is_seq_std:0.0014459878439083695 - rollout_corr/rollout_is_seq_max:1.005103349685669 - rollout_corr/rollout_is_seq_min:0.9960857629776001 - rollout_corr/rollout_is_seq_max_deviation:0.005103349685668945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2919784113764763) - rollout_corr/training_log_ppl:np.float64(0.806903613905888) - rollout_corr/kl:np.float64(0.0006409502570816628) - rollout_corr/k3_kl:np.float64(0.0006509173431368254) - rollout_corr/rollout_ppl:np.float64(2.290479706134647) - rollout_corr/rollout_log_ppl:np.float64(0.8062626618193462) - rollout_corr/log_ppl_diff:np.float64(0.0006409520865418017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012293330510146916) - rollout_corr/log_ppl_diff_max:np.float64(0.005213141441345215) - rollout_corr/log_ppl_diff_min:np.float64(-0.004137933254241943) - rollout_corr/ppl_ratio:np.float64(1.0006422218866646) - rollout_corr/chi2_token:np.float64(0.0013303321320563555) - rollout_corr/chi2_seq:np.float64(2.3223254224285483) - actor/pg_loss:np.float64(-0.0001239327248185873) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011024115792679368) - actor/ppo_kl:np.float64(6.963595266995526e-05) - actor/pg_clipfrac_lower:np.float64(2.784212483675219e-06) - actor/grad_norm:np.float64(0.24239107966423035) - perf/mfu/actor:np.float64(0.12592590460533137) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.14717817306519) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6589285714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.26147699146732906) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7634047619047619) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19852720557117806) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5552285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2397040092225637) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6592000000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.26073502970265483) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8421285714285713) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13617544486794253) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46132857142857137) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.20013008279957237) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6862619047619049) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20619876382539531) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8954952380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08197180786429441) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3817190476190476) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1582460634427849) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6993809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14750004177458542) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9245333333333334) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03779423823528531) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.31384761904761904) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.11509261056785722) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7040333333333333) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.108925053237339) - val-aux/sciknoweval/score/mean@16:np.float64(0.6589285714285714) - val-aux/sciknoweval/score/std@16:np.float64(0.26147699146732906) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7634047619047619) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19852720557117806) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5552285714285714) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2397040092225637) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6592000000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.26073502970265483) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8421285714285713) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13617544486794253) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46132857142857137) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.20013008279957237) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6862619047619049) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20619876382539531) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8954952380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08197180786429441) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3817190476190476) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1582460634427849) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6993809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14750004177458542) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9245333333333334) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03779423823528531) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.31384761904761904) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.11509261056785722) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7040333333333333) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.108925053237339) - val-core/sciknoweval/acc/mean@16:np.float64(0.6589285714285714) - val-aux/sciknoweval/acc/std@16:np.float64(0.26147699146732906) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7634047619047619) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19852720557117806) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5552285714285714) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2397040092225637) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6592000000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.26073502970265483) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8421285714285713) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13617544486794253) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46132857142857137) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.20013008279957237) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6862619047619049) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20619876382539531) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8954952380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08197180786429441) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3817190476190476) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1582460634427849) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6993809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14750004177458542) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9245333333333334) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03779423823528531) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.31384761904761904) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.11509261056785722) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7040333333333333) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.108925053237339) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.986904761904762) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.036673747734753556) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9979476190476191) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.012620842930515878) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9751857142857142) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04786735705451152) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9864428571428573) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.037432914347073416) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998619047619047) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0017967329312416017) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9577666666666667) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05639443888290546) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9949285714285714) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.022714201842546834) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9332380952380953) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05880143064108464) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9987619047619049) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009403419069561486) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9058904761904761) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.050590370282386964) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997761904761904) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002838983070955884) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001474093645811081 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001474093645811081 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:745.625 - response_length/max:1629.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:745.625 - response_length_non_aborted/max:1629.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00029304809868335724 - timing_s/agent_loop/generate_sequences/min:np.float64(2.268099818378687) - timing_s/agent_loop/generate_sequences/max:np.float64(14.966608583927155) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.466599339502864) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.966608583927155) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1629 - timing_s/gen:21.801297603175044 - timing_s/reward:0.08670702017843723 - timing_s/old_log_prob:3.37482875213027 - timing_s/adv:0.1377221718430519 - timing_s/update_actor:13.325021024793386 - timing_s/step:38.84823939576745 - timing_s/testing:189.37862921506166 - timing_s/save_checkpoint:12.842970063909888 - timing_s/stop_profile:0.00024105235934257507 - timing_per_token_ms/adv:0.0005318279728261195 - timing_per_token_ms/gen:0.11421467730079131 - timing_per_token_ms/update_actor:0.05145590448252003 - perf/total_num_tokens:258960 - perf/time_per_step:38.84823939576745 - perf/throughput:833.2423940820015 (TaskRunner pid=2224264) Training Progress: 60%|██████ | 60/100 [59:23<1:08:47, 103.19s/it] (TaskRunner pid=2224264) step:61 - global_seqlen/min:26866 - global_seqlen/max:38557 - global_seqlen/minmax_diff:11691 - global_seqlen/balanced_min:32591 - global_seqlen/balanced_max:32615 - global_seqlen/mean:32607.625 - actor/entropy:0.7484701871871948 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32022327184677124 - training/rollout_probs_diff_mean:0.004327286966145039 - training/rollout_probs_diff_std:0.008308346383273602 - training/rollout_actor_probs_pearson_corr:0.9996346235275269 - rollout_corr/rollout_is_mean:1.0000005960464478 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5481393347727135e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.3159535825252533 - rollout_corr/rollout_is_std:0.03432586044073105 - rollout_corr/rollout_is_eff_sample_size:0.9988231220557214 - rollout_corr/rollout_is_seq_mean:1.0000786781311035 - rollout_corr/rollout_is_seq_std:0.001447982620447874 - rollout_corr/rollout_is_seq_max:1.0040830373764038 - rollout_corr/rollout_is_seq_min:0.9964959025382996 - rollout_corr/rollout_is_seq_max_deviation:0.004083037376403809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.167487114202231) - rollout_corr/training_log_ppl:np.float64(0.7570779783418402) - rollout_corr/kl:np.float64(0.0007175489161515003) - rollout_corr/k3_kl:np.float64(0.0006208742135527245) - rollout_corr/rollout_ppl:np.float64(2.1659105462022126) - rollout_corr/rollout_log_ppl:np.float64(0.7563604272436351) - rollout_corr/log_ppl_diff:np.float64(0.0007175510982051492) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012418738333508372) - rollout_corr/log_ppl_diff_max:np.float64(0.007266104221343994) - rollout_corr/log_ppl_diff_min:np.float64(-0.003547847270965576) - rollout_corr/ppl_ratio:np.float64(1.0007189356256276) - rollout_corr/chi2_token:np.float64(0.0010508422274142504) - rollout_corr/chi2_seq:np.float64(1.2128563108853996) - actor/pg_loss:np.float64(7.9984194599092e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012339052045717835) - actor/ppo_kl:np.float64(0.0001892128098228696) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21570049226284027) - perf/mfu/actor:np.float64(0.1265814073644025) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.56413173675537) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024512207601219416 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024512207601219416 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:756.95703125 - response_length/max:1627.0 - response_length/min:211.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:756.95703125 - response_length_non_aborted/max:1627.0 - response_length_non_aborted/min:211.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018923170864582062 - timing_s/agent_loop/generate_sequences/min:np.float64(2.607853963971138) - timing_s/agent_loop/generate_sequences/max:np.float64(14.494692988693714) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.566406984020432) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.494692988693714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:1627 - timing_s/gen:20.789081823080778 - timing_s/reward:0.07305939309298992 - timing_s/old_log_prob:3.3369578402489424 - timing_s/adv:0.11469490267336369 - timing_s/update_actor:13.50451728142798 - timing_s/step:37.91396317072213 - timing_s/stop_profile:0.00010998733341693878 - timing_per_token_ms/adv:0.00043967822968310206 - timing_per_token_ms/gen:0.10728132181731324 - timing_per_token_ms/update_actor:0.05176901599483242 - perf/total_num_tokens:260861 - perf/time_per_step:37.91396317072213 - perf/throughput:860.0426405747057 (TaskRunner pid=2224264) Training Progress: 61%|██████ | 61/100 [1:00:01<54:21, 83.62s/it] (TaskRunner pid=2224264) step:62 - global_seqlen/min:27396 - global_seqlen/max:35660 - global_seqlen/minmax_diff:8264 - global_seqlen/balanced_min:31880 - global_seqlen/balanced_max:31902 - global_seqlen/mean:31893.125 - actor/entropy:0.8587515950202942 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2571102976799011 - training/rollout_probs_diff_mean:0.004673279356211424 - training/rollout_probs_diff_std:0.008435208350419998 - training/rollout_actor_probs_pearson_corr:0.9996413588523865 - rollout_corr/rollout_is_mean:0.9998844861984253 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9304524660110474 - rollout_corr/rollout_is_min:0.5585559010505676 - rollout_corr/rollout_is_std:0.03600391373038292 - rollout_corr/rollout_is_eff_sample_size:0.9987051585549779 - rollout_corr/rollout_is_seq_mean:0.9999316930770874 - rollout_corr/rollout_is_seq_std:0.0016182069666683674 - rollout_corr/rollout_is_seq_max:1.0045205354690552 - rollout_corr/rollout_is_seq_min:0.9909459948539734 - rollout_corr/rollout_is_seq_max_deviation:0.009054005146026611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.4564432627521455) - rollout_corr/training_log_ppl:np.float64(0.8728480438003317) - rollout_corr/kl:np.float64(0.000761049760463095) - rollout_corr/k3_kl:np.float64(0.000696839270858618) - rollout_corr/rollout_ppl:np.float64(2.454547442961484) - rollout_corr/rollout_log_ppl:np.float64(0.8720869926037267) - rollout_corr/log_ppl_diff:np.float64(0.0007610511966049671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013679289259016514) - rollout_corr/log_ppl_diff_max:np.float64(0.009840637445449829) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033975839614868164) - rollout_corr/ppl_ratio:np.float64(1.0007626954466105) - rollout_corr/chi2_token:np.float64(0.001287585822865367) - rollout_corr/chi2_seq:np.float64(1.209890850353986) - actor/pg_loss:np.float64(-3.718864172697067e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001223060548909416) - actor/ppo_kl:np.float64(2.9713928825003677e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29783353582024574) - perf/mfu/actor:np.float64(0.12234924411478575) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.59233093261719) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006781507283449173 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006781507283449173 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:697.41015625 - response_length/max:1847.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:697.41015625 - response_length_non_aborted/max:1847.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017420388758182526 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4971247743815184) - timing_s/agent_loop/generate_sequences/max:np.float64(15.592459442093968) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.807667781009513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.592459442093968) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:171 - timing_s/agent_loop/slowest/response_length:1847 - timing_s/gen:22.337838161736727 - timing_s/reward:0.07594001106917858 - timing_s/old_log_prob:3.2577473018318415 - timing_s/adv:0.1236040536314249 - timing_s/update_actor:13.773704160004854 - timing_s/step:39.65648138895631 - timing_s/stop_profile:0.0001120399683713913 - timing_per_token_ms/adv:0.00048444630947666974 - timing_per_token_ms/gen:0.12511601607362466 - timing_per_token_ms/update_actor:0.05398382943034296 - perf/total_num_tokens:255145 - perf/time_per_step:39.65648138895631 - perf/throughput:804.2348661039232 (TaskRunner pid=2224264) Training Progress: 62%|██████▏ | 62/100 [1:00:41<44:37, 70.45s/it] (TaskRunner pid=2224264) step:63 - global_seqlen/min:26919 - global_seqlen/max:40692 - global_seqlen/minmax_diff:13773 - global_seqlen/balanced_min:33436 - global_seqlen/balanced_max:33474 - global_seqlen/mean:33459.75 - actor/entropy:0.7692099809646606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975038170814514 - training/rollout_probs_diff_mean:0.0043107387609779835 - training/rollout_probs_diff_std:0.008506524376571178 - training/rollout_actor_probs_pearson_corr:0.9996923804283142 - rollout_corr/rollout_is_mean:0.999955415725708 - rollout_corr/rollout_is_ratio_fraction_high:5.143874204804888e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.143874204804888e-06 - rollout_corr/rollout_is_max:2.5493264198303223 - rollout_corr/rollout_is_min:0.3577776551246643 - rollout_corr/rollout_is_std:0.03481551632285118 - rollout_corr/rollout_is_eff_sample_size:0.9987891095585573 - rollout_corr/rollout_is_seq_mean:1.000030755996704 - rollout_corr/rollout_is_seq_std:0.0014692960539832711 - rollout_corr/rollout_is_seq_max:1.0053269863128662 - rollout_corr/rollout_is_seq_min:0.9961836934089661 - rollout_corr/rollout_is_seq_max_deviation:0.005326986312866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.23876611655578) - rollout_corr/training_log_ppl:np.float64(0.7817316737491637) - rollout_corr/kl:np.float64(0.0004539784086303733) - rollout_corr/k3_kl:np.float64(0.0006596663926075053) - rollout_corr/rollout_ppl:np.float64(2.2376751606352627) - rollout_corr/rollout_log_ppl:np.float64(0.781277688452974) - rollout_corr/log_ppl_diff:np.float64(0.000453985296189785) - rollout_corr/log_ppl_abs_diff:np.float64(0.001180113758891821) - rollout_corr/log_ppl_diff_max:np.float64(0.005360126495361328) - rollout_corr/log_ppl_diff_min:np.float64(-0.004343867301940918) - rollout_corr/ppl_ratio:np.float64(1.0004551277961582) - rollout_corr/chi2_token:np.float64(0.001740993233397603) - rollout_corr/chi2_seq:np.float64(1.6487001434434205) - actor/pg_loss:np.float64(-0.00019339460413902998) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018608193454383581) - actor/ppo_kl:np.float64(-0.00013630927301377938) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25826945155858994) - perf/mfu/actor:np.float64(0.12769393570296111) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.44427490234375) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002252373844385147 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002252373844385147 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.3984375 - response_length/max:1951.0 - response_length/min:194.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.3984375 - response_length_non_aborted/max:1951.0 - response_length_non_aborted/min:194.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010393001139163971 - timing_s/agent_loop/generate_sequences/min:np.float64(2.39320345595479) - timing_s/agent_loop/generate_sequences/max:np.float64(15.86801179125905) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.712352399430529) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.86801179125905) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:1951 - timing_s/gen:23.052989764139056 - timing_s/reward:0.07871297188103199 - timing_s/old_log_prob:3.3043668009340763 - timing_s/adv:0.11869384348392487 - timing_s/update_actor:13.733131846413016 - timing_s/step:40.37277623265982 - timing_s/stop_profile:0.00011059269309043884 - timing_per_token_ms/adv:0.0004434202417977005 - timing_per_token_ms/gen:0.1185816783645518 - timing_per_token_ms/update_actor:0.05130467145754607 - perf/total_num_tokens:267678 - perf/time_per_step:40.37277623265982 - perf/throughput:828.7701050623444 (TaskRunner pid=2224264) Training Progress: 63%|██████▎ | 63/100 [1:01:21<37:53, 61.44s/it] (TaskRunner pid=2224264) step:64 - global_seqlen/min:26306 - global_seqlen/max:34933 - global_seqlen/minmax_diff:8627 - global_seqlen/balanced_min:30904 - global_seqlen/balanced_max:30933 - global_seqlen/mean:30923.0 - actor/entropy:0.7835981845855713 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175749182701111 - training/rollout_probs_diff_mean:0.004405513871461153 - training/rollout_probs_diff_std:0.008360867388546467 - training/rollout_actor_probs_pearson_corr:0.9996961355209351 - rollout_corr/rollout_is_mean:0.9999273419380188 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1598237506404985e-05 - rollout_corr/rollout_is_max:1.7908501625061035 - rollout_corr/rollout_is_min:0.2933122217655182 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987892284793369 - rollout_corr/rollout_is_seq_mean:0.9998888373374939 - rollout_corr/rollout_is_seq_std:0.0014235756825655699 - rollout_corr/rollout_is_seq_max:1.0034641027450562 - rollout_corr/rollout_is_seq_min:0.9946786165237427 - rollout_corr/rollout_is_seq_max_deviation:0.005321383476257324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.228744763880968) - rollout_corr/training_log_ppl:np.float64(0.7778443787246943) - rollout_corr/kl:np.float64(0.000828439747124321) - rollout_corr/k3_kl:np.float64(0.0006999520201134146) - rollout_corr/rollout_ppl:np.float64(2.2268494172021747) - rollout_corr/rollout_log_ppl:np.float64(0.7770159327192232) - rollout_corr/log_ppl_diff:np.float64(0.0008284460054710507) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013873394345864654) - rollout_corr/log_ppl_diff_max:np.float64(0.007387757301330566) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033279061317443848) - rollout_corr/ppl_ratio:np.float64(1.0008300342597067) - rollout_corr/chi2_token:np.float64(0.0011283555068075657) - rollout_corr/chi2_seq:np.float64(1.5243452661670744) - actor/pg_loss:np.float64(-7.703946903347969e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002916122386977804) - actor/ppo_kl:np.float64(8.220684872384254e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.241146732121706) - perf/mfu/actor:np.float64(0.12080489767203606) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.29822540283203) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005282997153699398 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005282997153699398 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:673.59375 - response_length/max:1796.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:673.59375 - response_length_non_aborted/max:1796.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012784451246261597 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1317031513899565) - timing_s/agent_loop/generate_sequences/max:np.float64(14.859888169914484) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.550355381361442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.859888169914484) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1796 - timing_s/gen:21.599143993109465 - timing_s/reward:0.07637070119380951 - timing_s/old_log_prob:3.1958085894584656 - timing_s/adv:0.11453193984925747 - timing_s/update_actor:13.29977210238576 - timing_s/step:38.375933611765504 - timing_s/stop_profile:0.00011328049004077911 - timing_per_token_ms/adv:0.0004629723015605596 - timing_per_token_ms/gen:0.12525599624860512 - timing_per_token_ms/update_actor:0.053761650318475566 - perf/total_num_tokens:247384 - perf/time_per_step:38.375933611765504 - perf/throughput:805.7914711036361 (TaskRunner pid=2224264) Training Progress: 64%|██████▍ | 64/100 [1:01:59<32:43, 54.53s/it] (TaskRunner pid=2224264) step:65 - global_seqlen/min:24032 - global_seqlen/max:34111 - global_seqlen/minmax_diff:10079 - global_seqlen/balanced_min:29951 - global_seqlen/balanced_max:30559 - global_seqlen/mean:30032.625 - actor/entropy:0.7714234590530396 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28636491298675537 - training/rollout_probs_diff_mean:0.004574572201818228 - training/rollout_probs_diff_std:0.008538489229977131 - training/rollout_actor_probs_pearson_corr:0.9996505379676819 - rollout_corr/rollout_is_mean:0.9998939633369446 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6290990114212036 - rollout_corr/rollout_is_min:0.5353134870529175 - rollout_corr/rollout_is_std:0.03545675054192543 - rollout_corr/rollout_is_eff_sample_size:0.9987440406221719 - rollout_corr/rollout_is_seq_mean:0.9999128580093384 - rollout_corr/rollout_is_seq_std:0.0015794334467500448 - rollout_corr/rollout_is_seq_max:1.0056021213531494 - rollout_corr/rollout_is_seq_min:0.9935817122459412 - rollout_corr/rollout_is_seq_max_deviation:0.006418287754058838 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.22273992234841) - rollout_corr/training_log_ppl:np.float64(0.7840376188978553) - rollout_corr/kl:np.float64(0.0006947023282570797) - rollout_corr/k3_kl:np.float64(0.0007119409705182989) - rollout_corr/rollout_ppl:np.float64(2.221208442468196) - rollout_corr/rollout_log_ppl:np.float64(0.7833429137244821) - rollout_corr/log_ppl_diff:np.float64(0.0006947051733732224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013338688295334578) - rollout_corr/log_ppl_diff_max:np.float64(0.0072593092918396) - rollout_corr/log_ppl_diff_min:np.float64(-0.004377126693725586) - rollout_corr/ppl_ratio:np.float64(1.0006962057668716) - rollout_corr/chi2_token:np.float64(0.0014604993630200624) - rollout_corr/chi2_seq:np.float64(0.6011870829388499) - actor/pg_loss:np.float64(0.0001900325296446681) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003155937018846089) - actor/ppo_kl:np.float64(-3.965757715462814e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28370653092861176) - perf/mfu/actor:np.float64(0.11707659877392018) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.31705093383789) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018750634044408798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018750634044408798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:654.92578125 - response_length/max:2451.0 - response_length/min:231.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:654.92578125 - response_length_non_aborted/max:2451.0 - response_length_non_aborted/min:231.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011583603918552399 - timing_s/agent_loop/generate_sequences/min:np.float64(2.9298688657581806) - timing_s/agent_loop/generate_sequences/max:np.float64(18.389138467609882) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.613169322678004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.389138467609882) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:2451 - timing_s/gen:25.793642738834023 - timing_s/reward:0.07493023946881294 - timing_s/old_log_prob:3.18870216794312 - timing_s/adv:0.1171203050762415 - timing_s/update_actor:13.542755868285894 - timing_s/step:42.80827572196722 - timing_s/stop_profile:0.00011235475540161133 - timing_per_token_ms/adv:0.00048747114627942733 - timing_per_token_ms/gen:0.15384402299183486 - timing_per_token_ms/update_actor:0.0563668505012711 - perf/total_num_tokens:240261 - perf/time_per_step:42.80827572196722 - perf/throughput:701.5611933322662 (TaskRunner pid=2224264) Training Progress: 65%|██████▌ | 65/100 [1:02:42<29:46, 51.03s/it] (TaskRunner pid=2224264) step:66 - global_seqlen/min:28697 - global_seqlen/max:35089 - global_seqlen/minmax_diff:6392 - global_seqlen/balanced_min:30450 - global_seqlen/balanced_max:30469 - global_seqlen/mean:30460.5 - actor/entropy:0.7473804354667664 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424738645553589 - training/rollout_probs_diff_mean:0.004387002903968096 - training/rollout_probs_diff_std:0.008388126268982887 - training/rollout_actor_probs_pearson_corr:0.9996775388717651 - rollout_corr/rollout_is_mean:0.9999525547027588 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1807769624283537e-05 - rollout_corr/rollout_is_max:1.7755447626113892 - rollout_corr/rollout_is_min:0.3809736371040344 - rollout_corr/rollout_is_std:0.0348651260137558 - rollout_corr/rollout_is_eff_sample_size:0.9987857797882222 - rollout_corr/rollout_is_seq_mean:0.999984622001648 - rollout_corr/rollout_is_seq_std:0.001510056434199214 - rollout_corr/rollout_is_seq_max:1.0043302774429321 - rollout_corr/rollout_is_seq_min:0.9965217709541321 - rollout_corr/rollout_is_seq_max_deviation:0.004330277442932129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1832842687144876) - rollout_corr/training_log_ppl:np.float64(0.7604161555645987) - rollout_corr/kl:np.float64(0.0006697535459316839) - rollout_corr/k3_kl:np.float64(0.0006770667089313065) - rollout_corr/rollout_ppl:np.float64(2.18177503393963) - rollout_corr/rollout_log_ppl:np.float64(0.7597463995916769) - rollout_corr/log_ppl_diff:np.float64(0.0006697559729218483) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012636978644877672) - rollout_corr/log_ppl_diff_max:np.float64(0.005221843719482422) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034908056259155273) - rollout_corr/ppl_ratio:np.float64(1.0006710432935506) - rollout_corr/chi2_token:np.float64(0.001379573019221425) - rollout_corr/chi2_seq:np.float64(0.7963614012114704) - actor/pg_loss:np.float64(-9.32483235374093e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021860310857846343) - actor/ppo_kl:np.float64(3.254798675556003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2768869362771511) - perf/mfu/actor:np.float64(0.11916675434662388) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.34953784942627) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009460975416004658 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009460975416004658 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:661.640625 - response_length/max:1563.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:661.640625 - response_length_non_aborted/max:1563.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000106053426861763 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3329896610230207) - timing_s/agent_loop/generate_sequences/max:np.float64(13.444917742162943) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.728698585859092) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.444917742162943) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:1563 - timing_s/gen:20.854996770620346 - timing_s/reward:0.0732195358723402 - timing_s/old_log_prob:3.1821087524294853 - timing_s/adv:0.1175671610981226 - timing_s/update_actor:13.4094133451581 - timing_s/step:37.7182090356946 - timing_s/stop_profile:0.00011660158634185791 - timing_per_token_ms/adv:0.0004824574493939799 - timing_per_token_ms/gen:0.12312549752403086 - timing_per_token_ms/update_actor:0.05502787768240057 - perf/total_num_tokens:243684 - perf/time_per_step:37.7182090356946 - perf/throughput:807.5807621505498 (TaskRunner pid=2224264) Training Progress: 66%|██████▌ | 66/100 [1:03:20<26:39, 47.05s/it] (TaskRunner pid=2224264) step:67 - global_seqlen/min:21976 - global_seqlen/max:31921 - global_seqlen/minmax_diff:9945 - global_seqlen/balanced_min:28720 - global_seqlen/balanced_max:28745 - global_seqlen/mean:28734.25 - actor/entropy:0.7726788520812988 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3997594714164734 - training/rollout_probs_diff_mean:0.004599802196025848 - training/rollout_probs_diff_std:0.008748593740165234 - training/rollout_actor_probs_pearson_corr:0.9996384978294373 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9334631360834464e-05 - rollout_corr/rollout_is_max:1.7132740020751953 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03550714626908302 - rollout_corr/rollout_is_eff_sample_size:0.9987409489708702 - rollout_corr/rollout_is_seq_mean:1.0000040531158447 - rollout_corr/rollout_is_seq_std:0.0016247251769527793 - rollout_corr/rollout_is_seq_max:1.005348563194275 - rollout_corr/rollout_is_seq_min:0.9943971037864685 - rollout_corr/rollout_is_seq_max_deviation:0.005602896213531494 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.254453372210264) - rollout_corr/training_log_ppl:np.float64(0.7888535652309656) - rollout_corr/kl:np.float64(0.0006227922752195525) - rollout_corr/k3_kl:np.float64(0.0006862304701940047) - rollout_corr/rollout_ppl:np.float64(2.2529089520685375) - rollout_corr/rollout_log_ppl:np.float64(0.7882307717809454) - rollout_corr/log_ppl_diff:np.float64(0.0006227934500202537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014135913224890828) - rollout_corr/log_ppl_diff_max:np.float64(0.007272124290466309) - rollout_corr/log_ppl_diff_min:np.float64(-0.004362225532531738) - rollout_corr/ppl_ratio:np.float64(1.000624400563538) - rollout_corr/chi2_token:np.float64(0.001499387202784419) - rollout_corr/chi2_seq:np.float64(1.2466026986949146) - actor/pg_loss:np.float64(-2.0926236175000668e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024252940147562185) - actor/ppo_kl:np.float64(-2.8959387931237757e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29370710253715515) - perf/mfu/actor:np.float64(0.11386412359551698) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.32535934448242) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009934777393937111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009934777393937111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:606.1015625 - response_length/max:1545.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:606.1015625 - response_length_non_aborted/max:1545.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00022127665579319 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8514165561646223) - timing_s/agent_loop/generate_sequences/max:np.float64(13.32896644063294) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.998300799677963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.32896644063294) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:1545 - timing_s/gen:20.499601708725095 - timing_s/reward:0.07158185541629791 - timing_s/old_log_prob:3.1310275867581367 - timing_s/adv:0.12111156806349754 - timing_s/update_actor:13.336501782760024 - timing_s/step:37.24595209956169 - timing_s/stop_profile:0.00011993572115898132 - timing_per_token_ms/adv:0.0005268606630741082 - timing_per_token_ms/gen:0.1321174108913593 - timing_per_token_ms/update_actor:0.058016573352184346 - perf/total_num_tokens:229874 - perf/time_per_step:37.24595209956169 - perf/throughput:771.4730965445812 (TaskRunner pid=2224264) Training Progress: 67%|██████▋ | 67/100 [1:03:57<24:16, 44.12s/it] (TaskRunner pid=2224264) step:68 - global_seqlen/min:23879 - global_seqlen/max:35240 - global_seqlen/minmax_diff:11361 - global_seqlen/balanced_min:29992 - global_seqlen/balanced_max:30029 - global_seqlen/mean:30017.25 - actor/entropy:0.7595304250717163 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7168671488761902 - training/rollout_probs_diff_mean:0.004404507111757994 - training/rollout_probs_diff_std:0.008633471094071865 - training/rollout_actor_probs_pearson_corr:0.9996846318244934 - rollout_corr/rollout_is_mean:0.999897301197052 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8815619114320725e-05 - rollout_corr/rollout_is_max:1.5993539094924927 - rollout_corr/rollout_is_min:0.019239548593759537 - rollout_corr/rollout_is_std:0.03493344411253929 - rollout_corr/rollout_is_eff_sample_size:0.9987809040931451 - rollout_corr/rollout_is_seq_mean:0.9998229742050171 - rollout_corr/rollout_is_seq_std:0.0016775608528405428 - rollout_corr/rollout_is_seq_max:1.005442500114441 - rollout_corr/rollout_is_seq_min:0.9936239123344421 - rollout_corr/rollout_is_seq_max_deviation:0.006376087665557861 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.246089845895767) - rollout_corr/training_log_ppl:np.float64(0.7848675699206069) - rollout_corr/kl:np.float64(0.0009016538949424557) - rollout_corr/k3_kl:np.float64(0.0007493318355500378) - rollout_corr/rollout_ppl:np.float64(2.243979589547962) - rollout_corr/rollout_log_ppl:np.float64(0.7839659146266058) - rollout_corr/log_ppl_diff:np.float64(0.0009016552940011024) - rollout_corr/log_ppl_abs_diff:np.float64(0.001556948758661747) - rollout_corr/log_ppl_diff_max:np.float64(0.012223243713378906) - rollout_corr/log_ppl_diff_min:np.float64(-0.004822075366973877) - rollout_corr/ppl_ratio:np.float64(1.0009038699790835) - rollout_corr/chi2_token:np.float64(0.0011472513433545828) - rollout_corr/chi2_seq:np.float64(2.487598191248253) - actor/pg_loss:np.float64(-4.085572436451912e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027236666869612236) - actor/ppo_kl:np.float64(5.479786341200388e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2867882438004017) - perf/mfu/actor:np.float64(0.11808746991936632) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.29296493530273) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.41796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006193474982865155 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006193474982865155 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:622.8203125 - response_length/max:1786.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:622.8203125 - response_length_non_aborted/max:1786.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014729611575603485 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0367041416466236) - timing_s/agent_loop/generate_sequences/max:np.float64(14.751183833926916) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.172361494318466) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.751183833926916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:1786 - timing_s/gen:22.098993657156825 - timing_s/reward:0.10232115164399147 - timing_s/old_log_prob:3.2531425300985575 - timing_s/adv:0.1175235714763403 - timing_s/update_actor:13.300975374877453 - timing_s/step:38.95930672250688 - timing_s/stop_profile:0.00012533552944660187 - timing_per_token_ms/adv:0.0004894001427360113 - timing_per_token_ms/gen:0.13860208512911795 - timing_per_token_ms/update_actor:0.05538888212143623 - perf/total_num_tokens:240138 - perf/time_per_step:38.95930672250688 - perf/throughput:770.4770060155862 (TaskRunner pid=2224264) Training Progress: 68%|██████▊ | 68/100 [1:04:36<22:42, 42.59s/it] (TaskRunner pid=2224264) step:69 - global_seqlen/min:24189 - global_seqlen/max:45366 - global_seqlen/minmax_diff:21177 - global_seqlen/balanced_min:33420 - global_seqlen/balanced_max:39649 - global_seqlen/mean:34239.875 - actor/entropy:0.7071983814239502 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38365769386291504 - training/rollout_probs_diff_mean:0.0040749297477304935 - training/rollout_probs_diff_std:0.00811900943517685 - training/rollout_actor_probs_pearson_corr:0.9996503591537476 - rollout_corr/rollout_is_mean:1.000025749206543 - rollout_corr/rollout_is_ratio_fraction_high:5.1385613915044814e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.1385613915044814e-06 - rollout_corr/rollout_is_max:2.4045958518981934 - rollout_corr/rollout_is_min:0.4110872149467468 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988547581358254 - rollout_corr/rollout_is_seq_mean:1.0000370740890503 - rollout_corr/rollout_is_seq_std:0.0013557960046455264 - rollout_corr/rollout_is_seq_max:1.003797173500061 - rollout_corr/rollout_is_seq_min:0.9964458346366882 - rollout_corr/rollout_is_seq_max_deviation:0.003797173500061035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.177241276949644) - rollout_corr/training_log_ppl:np.float64(0.7569285913486965) - rollout_corr/kl:np.float64(0.0004947985197105709) - rollout_corr/k3_kl:np.float64(0.0006430674019384242) - rollout_corr/rollout_ppl:np.float64(2.1761399055831134) - rollout_corr/rollout_log_ppl:np.float64(0.7564337924704887) - rollout_corr/log_ppl_diff:np.float64(0.0004947988782078028) - rollout_corr/log_ppl_abs_diff:np.float64(0.001128945965319872) - rollout_corr/log_ppl_diff_max:np.float64(0.004031240940093994) - rollout_corr/log_ppl_diff_min:np.float64(-0.004697918891906738) - rollout_corr/ppl_ratio:np.float64(1.00049584871158) - rollout_corr/chi2_token:np.float64(0.001590353436768055) - rollout_corr/chi2_seq:np.float64(0.8412239120807499) - actor/pg_loss:np.float64(-7.094477768987417e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019783686673235934) - actor/ppo_kl:np.float64(-9.313821129719457e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25627975165843964) - perf/mfu/actor:np.float64(0.12214039553244277) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.25387191772461) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010957982391119003 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010957982391119003 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:760.18359375 - response_length/max:8192.0 - response_length/min:169.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:760.18359375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012904033064842224 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1227447520941496) - timing_s/agent_loop/generate_sequences/max:np.float64(60.78352793678641) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.487850829478703) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.78352793678641) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:597 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.3937315158546 - timing_s/reward:0.07918412238359451 - timing_s/old_log_prob:3.5191430747509003 - timing_s/adv:0.11993566527962685 - timing_s/update_actor:14.636007921770215 - timing_s/step:85.83626089058816 - timing_s/stop_profile:0.00012475252151489258 - timing_per_token_ms/adv:0.00043785084378822515 - timing_per_token_ms/gen:0.3463068210077469 - timing_per_token_ms/update_actor:0.0534318828623433 - perf/total_num_tokens:273919 - perf/time_per_step:85.83626089058816 - perf/throughput:398.8975596647216 (TaskRunner pid=2224264) Training Progress: 69%|██████▉ | 69/100 [1:06:02<28:43, 55.58s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a molecular structure with the following key features: (TaskRunner pid=2224264) - A nitrogen atom at the end of a chain (Nc1...n1) (TaskRunner pid=2224264) - A benzene ring with a substituent: `Cc2ccc(Cl)cc2C(F)(F)F` (i.e., a substituted benzene ring with a chloro group and a trifluoromethyl group) (TaskRunner pid=2224264) - The overall structure appears to be a **pyrrole-like ring** (Nc1ccn...n1) with a substituent group on the nitrogen. (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are to identify which of the four options is the **only correct reactant** that would be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This contains the core structure: `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` — which matches the product. (TaskRunner pid=2224264) - It also has a ketone group `O=C1` and another ketone group `C(=O)N1` — which may be a part of a synthesis pathway (e.g., a diketone or acylated structure). (TaskRunner pid=2224264) - This seems like a plausible reactant that could be used to form the product via a reaction like a **condensation** or **cyclization**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This contains a `C(Cl)` group, not the `C(F)(F)F` group we see in the product. (TaskRunner pid=2224264) - The rest of the structure is not matching the product's ring system or substituents. (TaskRunner pid=2224264) - Therefore, this is **not a correct reactant**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a `C(F)(F)F` group (trifluoromethyl), which matches the product. (TaskRunner pid=2224264) - It also has a `Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` — which includes a benzene ring with a chloro and trifluoromethyl group. (TaskRunner pid=2224264) - However, this structure is more of a **complex aromatic ring system**, not a direct match for the pyrrole-like structure in the product. (TaskRunner pid=2224264) - It may be a reactant in a more complex multi-step synthesis, but it is **not the only correct one**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains the `C(F)(F)F` group and a benzene ring. (TaskRunner pid=2224264) - However, it does **not** match the nitrogen-containing ring structure of the product. (TaskRunner pid=2224264) - The presence of a double bond in the side chain (CC=C) is not matching the pyrrole-like ring in the product. (TaskRunner pid=2224264) - So this is **not the correct reactant**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Conclusion: (TaskRunner pid=2224264) Only **Option A** contains the **exact ring structure** and **substituents** needed to form the product. It includes the pyrrole-like ring (`c1ccn(...n1`), the chloro-substituted benzene ring with a trifluoromethyl group, and has functional groups that are consistent with a synthetic pathway leading to the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2224264) step:70 - global_seqlen/min:22032 - global_seqlen/max:35660 - global_seqlen/minmax_diff:13628 - global_seqlen/balanced_min:30368 - global_seqlen/balanced_max:30409 - global_seqlen/mean:30397.25 - actor/entropy:0.8035950064659119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22990918159484863 - training/rollout_probs_diff_mean:0.004655633121728897 - training/rollout_probs_diff_std:0.008662798441946507 - training/rollout_actor_probs_pearson_corr:0.9996584057807922 - rollout_corr/rollout_is_mean:1.0000817775726318 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.9939820857835e-06 - rollout_corr/rollout_is_max:1.8103430271148682 - rollout_corr/rollout_is_min:0.4562520384788513 - rollout_corr/rollout_is_std:0.035914406180381775 - rollout_corr/rollout_is_eff_sample_size:0.9987120548475574 - rollout_corr/rollout_is_seq_mean:1.0000861883163452 - rollout_corr/rollout_is_seq_std:0.0016128499992191792 - rollout_corr/rollout_is_seq_max:1.00638747215271 - rollout_corr/rollout_is_seq_min:0.9938380718231201 - rollout_corr/rollout_is_seq_max_deviation:0.006387472152709961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.279810177627951) - rollout_corr/training_log_ppl:np.float64(0.8013737860601395) - rollout_corr/kl:np.float64(0.0007208169924197705) - rollout_corr/k3_kl:np.float64(0.0007489672824476656) - rollout_corr/rollout_ppl:np.float64(2.278083552606404) - rollout_corr/rollout_log_ppl:np.float64(0.8006529699778184) - rollout_corr/log_ppl_diff:np.float64(0.0007208160823211074) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014672434190288186) - rollout_corr/log_ppl_diff_max:np.float64(0.008777737617492676) - rollout_corr/log_ppl_diff_min:np.float64(-0.005332350730895996) - rollout_corr/ppl_ratio:np.float64(1.000722888391465) - rollout_corr/chi2_token:np.float64(0.0015351453330367804) - rollout_corr/chi2_seq:np.float64(2.378258903743699) - actor/pg_loss:np.float64(-0.000118100899271667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018792178138937743) - actor/ppo_kl:np.float64(0.00014457460659045296) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2343890555202961) - perf/mfu/actor:np.float64(0.11821289785724132) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.17000198364258) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6714285714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.2317466189895333) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7653619047619048) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1782119464541067) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5797619047619048) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2136060282647753) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6726619047619047) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.23202999913590713) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8335190476190476) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11938512568929552) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49611428571428573) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1773846797198801) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6981047619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.184153742497439) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8789999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0730800489542387) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42429047619047616) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.13809200842124467) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7115333333333334) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13558200983257696) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9054095238095239) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03841748334380248) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3662142857142857) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.09495676645373348) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7192904761904763) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1010536286928432) - val-aux/sciknoweval/score/mean@16:np.float64(0.6714285714285714) - val-aux/sciknoweval/score/std@16:np.float64(0.2317466189895333) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7653619047619048) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1782119464541067) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5797619047619048) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2136060282647753) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6726619047619047) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.23202999913590713) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8335190476190476) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11938512568929552) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49611428571428573) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1773846797198801) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6981047619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.184153742497439) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8789999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0730800489542387) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42429047619047616) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.13809200842124467) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7115333333333334) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13558200983257696) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9054095238095239) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03841748334380248) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3662142857142857) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.09495676645373348) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7192904761904763) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1010536286928432) - val-core/sciknoweval/acc/mean@16:np.float64(0.6714285714285714) - val-aux/sciknoweval/acc/std@16:np.float64(0.2317466189895333) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7653619047619048) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1782119464541067) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5797619047619048) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2136060282647753) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6726619047619047) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.23202999913590713) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8335190476190476) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11938512568929552) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49611428571428573) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1773846797198801) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6981047619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.184153742497439) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8789999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0730800489542387) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42429047619047616) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.13809200842124467) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7115333333333334) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13558200983257696) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9054095238095239) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03841748334380248) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3662142857142857) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.09495676645373348) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7192904761904763) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1010536286928432) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9851190476190477) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.044118643209065715) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9977809523809523) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.015483180068832298) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9722142857142857) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.057096894053364385) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9851714285714287) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04372250124311388) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998476190476191) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.001196522485392713) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9513904761904762) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0690690442763583) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9942000000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.027176785822535302) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.920295238095238) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07418068438997008) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9985380952380953) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.012138625400942288) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8851476190476191) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06448749000951351) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998714285714286) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002292661410582752) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01194525696337223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01194525696337223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:651.6953125 - response_length/max:1845.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:651.6953125 - response_length_non_aborted/max:1845.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011080317199230194 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8444180563092232) - timing_s/agent_loop/generate_sequences/max:np.float64(14.495014188811183) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.144595855919761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.495014188811183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1676 - timing_s/gen:21.431634962558746 - timing_s/reward:0.07384154014289379 - timing_s/old_log_prob:3.262227326631546 - timing_s/adv:0.11903642117977142 - timing_s/update_actor:13.540440015494823 - timing_s/step:38.51284538023174 - timing_s/testing:189.72432128712535 - timing_s/save_checkpoint:13.012803422287107 - timing_s/stop_profile:0.00014807283878326416 - timing_per_token_ms/adv:0.0004895032493883963 - timing_per_token_ms/gen:0.12846083509691517 - timing_per_token_ms/update_actor:0.05568118832910388 - perf/total_num_tokens:243178 - perf/time_per_step:38.51284538023174 - perf/throughput:789.2756221954612 (TaskRunner pid=2224264) Training Progress: 70%|███████ | 70/100 [1:10:04<55:39, 111.31s/it] (TaskRunner pid=2224264) step:71 - global_seqlen/min:26528 - global_seqlen/max:38219 - global_seqlen/minmax_diff:11691 - global_seqlen/balanced_min:31790 - global_seqlen/balanced_max:32177 - global_seqlen/mean:31863.5 - actor/entropy:0.7618645429611206 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21302002668380737 - training/rollout_probs_diff_mean:0.004489706829190254 - training/rollout_probs_diff_std:0.008394083939492702 - training/rollout_actor_probs_pearson_corr:0.9996015429496765 - rollout_corr/rollout_is_mean:0.9997878670692444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.690140925347805e-05 - rollout_corr/rollout_is_max:1.6874182224273682 - rollout_corr/rollout_is_min:0.4357602596282959 - rollout_corr/rollout_is_std:0.034910403192043304 - rollout_corr/rollout_is_eff_sample_size:0.9987822122017861 - rollout_corr/rollout_is_seq_mean:0.9997093677520752 - rollout_corr/rollout_is_seq_std:0.0016728744376450777 - rollout_corr/rollout_is_seq_max:1.005326747894287 - rollout_corr/rollout_is_seq_min:0.9935998916625977 - rollout_corr/rollout_is_seq_max_deviation:0.006400108337402344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.216639978811145) - rollout_corr/training_log_ppl:np.float64(0.7745629880810156) - rollout_corr/kl:np.float64(0.000979817594189747) - rollout_corr/k3_kl:np.float64(0.0007290477265087247) - rollout_corr/rollout_ppl:np.float64(2.2143593542277813) - rollout_corr/rollout_log_ppl:np.float64(0.7735831652535126) - rollout_corr/log_ppl_diff:np.float64(0.0009798228275030851) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014419290237128735) - rollout_corr/log_ppl_diff_max:np.float64(0.008339285850524902) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046994686126708984) - rollout_corr/ppl_ratio:np.float64(1.0009817637037486) - rollout_corr/chi2_token:np.float64(0.0009503676556050777) - rollout_corr/chi2_seq:np.float64(0.568886311724782) - actor/pg_loss:np.float64(5.502370186150074e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019256132691225503) - actor/ppo_kl:np.float64(4.591261260600277e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20790256932377815) - perf/mfu/actor:np.float64(0.12276829314252474) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.04192352294922) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034556337632238865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034556337632238865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:693.359375 - response_length/max:2588.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:693.359375 - response_length_non_aborted/max:2588.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.34839928150177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.169468689709902) - timing_s/agent_loop/generate_sequences/max:np.float64(19.67856295593083) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.767557479441166) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.67856295593083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:2588 - timing_s/gen:26.15543474815786 - timing_s/reward:0.07458089478313923 - timing_s/old_log_prob:3.3945595882833004 - timing_s/adv:0.11478821560740471 - timing_s/update_actor:13.66449068300426 - timing_s/step:43.516275111585855 - timing_s/stop_profile:3.4091994166374207e-05 - timing_per_token_ms/adv:0.0004503123307522899 - timing_per_token_ms/gen:0.14735456196145272 - timing_per_token_ms/update_actor:0.05360557802424506 - perf/total_num_tokens:254908 - perf/time_per_step:43.516275111585855 - perf/throughput:732.220299607321 (TaskRunner pid=2224264) Training Progress: 71%|███████ | 71/100 [1:10:47<43:58, 90.98s/it] (TaskRunner pid=2224264) step:72 - global_seqlen/min:26255 - global_seqlen/max:35299 - global_seqlen/minmax_diff:9044 - global_seqlen/balanced_min:30687 - global_seqlen/balanced_max:30704 - global_seqlen/mean:30699.25 - actor/entropy:0.7660148739814758 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2309356927871704 - training/rollout_probs_diff_mean:0.0045791687443852425 - training/rollout_probs_diff_std:0.008554509840905666 - training/rollout_actor_probs_pearson_corr:0.999636709690094 - rollout_corr/rollout_is_mean:0.9999493360519409 - rollout_corr/rollout_is_ratio_fraction_high:5.8084829106519464e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0826847553253174 - rollout_corr/rollout_is_min:0.5040711760520935 - rollout_corr/rollout_is_std:0.03556250035762787 - rollout_corr/rollout_is_eff_sample_size:0.9987369060711254 - rollout_corr/rollout_is_seq_mean:0.9999403953552246 - rollout_corr/rollout_is_seq_std:0.0016658001113682985 - rollout_corr/rollout_is_seq_max:1.006662368774414 - rollout_corr/rollout_is_seq_min:0.99498051404953 - rollout_corr/rollout_is_seq_max_deviation:0.0066623687744140625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.172296098433435) - rollout_corr/training_log_ppl:np.float64(0.7561264133546501) - rollout_corr/kl:np.float64(0.0008874626383423845) - rollout_corr/k3_kl:np.float64(0.0007379499492117247) - rollout_corr/rollout_ppl:np.float64(2.170314473565668) - rollout_corr/rollout_log_ppl:np.float64(0.7552389547927305) - rollout_corr/log_ppl_diff:np.float64(0.0008874585619196296) - rollout_corr/log_ppl_abs_diff:np.float64(0.001500150072388351) - rollout_corr/log_ppl_diff_max:np.float64(0.00498625636100769) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046173930168151855) - rollout_corr/ppl_ratio:np.float64(1.0008892305195332) - rollout_corr/chi2_token:np.float64(0.0011937415692955256) - rollout_corr/chi2_seq:np.float64(0.66025594715029) - actor/pg_loss:np.float64(3.53293726220727e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002114741264449549) - actor/ppo_kl:np.float64(0.00018003375489444862) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2194691300392151) - perf/mfu/actor:np.float64(0.1192681330197729) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.72951126098633) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007243903819471598 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007243903819471598 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:672.5078125 - response_length/max:1734.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:672.5078125 - response_length_non_aborted/max:1734.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6632100343704224e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.31886431761086) - timing_s/agent_loop/generate_sequences/max:np.float64(14.253411900252104) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.534527560528659) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.253411900252104) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:1734 - timing_s/gen:21.547888031229377 - timing_s/reward:0.07392375729978085 - timing_s/old_log_prob:3.2580875027924776 - timing_s/adv:0.11826586164534092 - timing_s/update_actor:13.605389777570963 - timing_s/step:38.68964970111847 - timing_s/stop_profile:0.00017290934920310974 - timing_per_token_ms/adv:0.00048155028887245175 - timing_per_token_ms/gen:0.12516053502648306 - timing_per_token_ms/update_actor:0.055397891550978295 - perf/total_num_tokens:245594 - perf/time_per_step:38.68964970111847 - perf/throughput:793.4744883232304 (TaskRunner pid=2224264) Training Progress: 72%|███████▏ | 72/100 [1:11:26<35:08, 75.31s/it] (TaskRunner pid=2224264) step:73 - global_seqlen/min:32171 - global_seqlen/max:38705 - global_seqlen/minmax_diff:6534 - global_seqlen/balanced_min:33645 - global_seqlen/balanced_max:39741 - global_seqlen/mean:34425.125 - actor/entropy:0.763630211353302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30194756388664246 - training/rollout_probs_diff_mean:0.004307533614337444 - training/rollout_probs_diff_std:0.008280006237328053 - training/rollout_actor_probs_pearson_corr:0.999650776386261 - rollout_corr/rollout_is_mean:0.9999016523361206 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.910024017590331e-06 - rollout_corr/rollout_is_max:1.6241782903671265 - rollout_corr/rollout_is_min:0.4921497702598572 - rollout_corr/rollout_is_std:0.034490425139665604 - rollout_corr/rollout_is_eff_sample_size:0.9988115860890442 - rollout_corr/rollout_is_seq_mean:0.9998833537101746 - rollout_corr/rollout_is_seq_std:0.0013762600719928741 - rollout_corr/rollout_is_seq_max:1.0045596361160278 - rollout_corr/rollout_is_seq_min:0.9957677125930786 - rollout_corr/rollout_is_seq_max_deviation:0.004559636116027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2688772692345083) - rollout_corr/training_log_ppl:np.float64(0.8037596073554596) - rollout_corr/kl:np.float64(0.0007620832186034932) - rollout_corr/k3_kl:np.float64(0.0006744360291293106) - rollout_corr/rollout_ppl:np.float64(2.267069778870791) - rollout_corr/rollout_log_ppl:np.float64(0.8029975256649777) - rollout_corr/log_ppl_diff:np.float64(0.0007620816904818639) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012150325201218948) - rollout_corr/log_ppl_diff_max:np.float64(0.006102800369262695) - rollout_corr/log_ppl_diff_min:np.float64(-0.003519415855407715) - rollout_corr/ppl_ratio:np.float64(1.0007633492350578) - rollout_corr/chi2_token:np.float64(0.0011786054819822311) - rollout_corr/chi2_seq:np.float64(0.6752500396687537) - actor/pg_loss:np.float64(7.905624806880951e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014205676188794314) - actor/ppo_kl:np.float64(1.4047067626421494e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2183140367269516) - perf/mfu/actor:np.float64(0.12278116839994453) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.55410766601562) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005568580701947212 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005568580701947212 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:795.56640625 - response_length/max:8192.0 - response_length/min:236.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:795.56640625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:236.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014955177903175354 - timing_s/agent_loop/generate_sequences/min:np.float64(2.818276045843959) - timing_s/agent_loop/generate_sequences/max:np.float64(61.54518798738718) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.143103411246557) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(61.54518798738718) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:68.7704401165247 - timing_s/reward:0.0806842241436243 - timing_s/old_log_prob:3.6475528068840504 - timing_s/adv:0.12522803619503975 - timing_s/update_actor:14.82589746080339 - timing_s/step:87.5357283577323 - timing_s/stop_profile:0.00011074915528297424 - timing_per_token_ms/adv:0.0004547116248490011 - timing_per_token_ms/gen:0.3376644986449547 - timing_per_token_ms/update_actor:0.053833854854569846 - perf/total_num_tokens:275401 - perf/time_per_step:87.5357283577323 - perf/throughput:393.26941862315726 (TaskRunner pid=2224264) Training Progress: 73%|███████▎ | 73/100 [1:12:54<35:32, 78.99s/it] (TaskRunner pid=2224264) step:74 - global_seqlen/min:29986 - global_seqlen/max:43985 - global_seqlen/minmax_diff:13999 - global_seqlen/balanced_min:33665 - global_seqlen/balanced_max:33695 - global_seqlen/mean:33681.375 - actor/entropy:0.761839747428894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19873207807540894 - training/rollout_probs_diff_mean:0.004497237037867308 - training/rollout_probs_diff_std:0.008453729562461376 - training/rollout_actor_probs_pearson_corr:0.999616265296936 - rollout_corr/rollout_is_mean:0.9999936819076538 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6013860702514648 - rollout_corr/rollout_is_min:0.606749951839447 - rollout_corr/rollout_is_std:0.03470577299594879 - rollout_corr/rollout_is_eff_sample_size:0.9987970773134115 - rollout_corr/rollout_is_seq_mean:0.9999818801879883 - rollout_corr/rollout_is_seq_std:0.0015479137655347586 - rollout_corr/rollout_is_seq_max:1.0074827671051025 - rollout_corr/rollout_is_seq_min:0.9962047338485718 - rollout_corr/rollout_is_seq_max_deviation:0.007482767105102539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2138938223943114) - rollout_corr/training_log_ppl:np.float64(0.7749610797036439) - rollout_corr/kl:np.float64(0.0006632767860637601) - rollout_corr/k3_kl:np.float64(0.0006568010118712664) - rollout_corr/rollout_ppl:np.float64(2.212330491282046) - rollout_corr/rollout_log_ppl:np.float64(0.7742978031164967) - rollout_corr/log_ppl_diff:np.float64(0.0006632765871472657) - rollout_corr/log_ppl_abs_diff:np.float64(0.001370829122606665) - rollout_corr/log_ppl_diff_max:np.float64(0.004554629325866699) - rollout_corr/log_ppl_diff_min:np.float64(-0.006599485874176025) - rollout_corr/ppl_ratio:np.float64(1.0006648083217442) - rollout_corr/chi2_token:np.float64(0.0013132649473845959) - rollout_corr/chi2_seq:np.float64(1.9479922475293279) - actor/pg_loss:np.float64(-5.3111580200493336e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014415476721296727) - actor/ppo_kl:np.float64(1.6327075574196215e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20973877236247063) - perf/mfu/actor:np.float64(0.12778629328477945) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.42087173461914) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005659057758748531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005659057758748531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:779.13671875 - response_length/max:2249.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:779.13671875 - response_length_non_aborted/max:2249.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010413303971290588 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6374386623501778) - timing_s/agent_loop/generate_sequences/max:np.float64(18.268491968512535) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.814450513171323) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.268491968512535) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:2249 - timing_s/gen:25.160439206287265 - timing_s/reward:0.07928537577390671 - timing_s/old_log_prob:3.3276401832699776 - timing_s/adv:0.12078331597149372 - timing_s/update_actor:13.96530625037849 - timing_s/step:42.73991168104112 - timing_s/stop_profile:0.00011014938354492188 - timing_per_token_ms/adv:0.0004482570707531006 - timing_per_token_ms/gen:0.12614341396621492 - timing_per_token_ms/update_actor:0.05182874159078456 - perf/total_num_tokens:269451 - perf/time_per_step:42.73991168104112 - perf/throughput:788.0543893341883 (TaskRunner pid=2224264) Training Progress: 74%|███████▍ | 74/100 [1:13:36<29:31, 68.13s/it] (TaskRunner pid=2224264) step:75 - global_seqlen/min:24487 - global_seqlen/max:40274 - global_seqlen/minmax_diff:15787 - global_seqlen/balanced_min:34105 - global_seqlen/balanced_max:34135 - global_seqlen/mean:34124.625 - actor/entropy:0.8144033551216125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917940020561218 - training/rollout_probs_diff_mean:0.00457530515268445 - training/rollout_probs_diff_std:0.008413956500589848 - training/rollout_actor_probs_pearson_corr:0.9996695518493652 - rollout_corr/rollout_is_mean:1.0000370740890503 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.187879196455469e-06 - rollout_corr/rollout_is_max:1.7824959754943848 - rollout_corr/rollout_is_min:0.4731596112251282 - rollout_corr/rollout_is_std:0.035698000341653824 - rollout_corr/rollout_is_eff_sample_size:0.9987273934949555 - rollout_corr/rollout_is_seq_mean:1.0000509023666382 - rollout_corr/rollout_is_seq_std:0.0015275044133886695 - rollout_corr/rollout_is_seq_max:1.004511833190918 - rollout_corr/rollout_is_seq_min:0.9952264428138733 - rollout_corr/rollout_is_seq_max_deviation:0.004773557186126709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2711348473094404) - rollout_corr/training_log_ppl:np.float64(0.803941803984344) - rollout_corr/kl:np.float64(0.0005384247337580916) - rollout_corr/k3_kl:np.float64(0.000659077487966897) - rollout_corr/rollout_ppl:np.float64(2.2699187486432493) - rollout_corr/rollout_log_ppl:np.float64(0.803403376834467) - rollout_corr/log_ppl_diff:np.float64(0.0005384271498769522) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012698217760771513) - rollout_corr/log_ppl_diff_max:np.float64(0.0053713321685791016) - rollout_corr/log_ppl_diff_min:np.float64(-0.003410249948501587) - rollout_corr/ppl_ratio:np.float64(1.0005397242493927) - rollout_corr/chi2_token:np.float64(0.0015677101910114288) - rollout_corr/chi2_seq:np.float64(1.5271832724101841) - actor/pg_loss:np.float64(-4.3374369852244854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.082515417598188e-05) - actor/ppo_kl:np.float64(-5.413583712154946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17228768952190876) - perf/mfu/actor:np.float64(0.13115778980210233) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.54808044433594) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018546667415648699 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018546667415648699 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:752.95703125 - response_length/max:1483.0 - response_length/min:199.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:752.95703125 - response_length_non_aborted/max:1483.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011150166392326355 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5929522551596165) - timing_s/agent_loop/generate_sequences/max:np.float64(14.319374568760395) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.897164289628563) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.319374568760395) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:1483 - timing_s/gen:21.023408722132444 - timing_s/reward:0.08007885701954365 - timing_s/old_log_prob:3.324468068778515 - timing_s/adv:0.12420106492936611 - timing_s/update_actor:13.739917827770114 - timing_s/step:38.37722675316036 - timing_s/stop_profile:0.00010982714593410492 - timing_per_token_ms/adv:0.0004549539552792379 - timing_per_token_ms/gen:0.109066901446549 - timing_per_token_ms/update_actor:0.050329922408561684 - perf/total_num_tokens:272997 - perf/time_per_step:38.37722675316036 - perf/throughput:889.189446113113 (TaskRunner pid=2224264) Training Progress: 75%|███████▌ | 75/100 [1:14:15<24:40, 59.22s/it] (TaskRunner pid=2224264) step:76 - global_seqlen/min:31915 - global_seqlen/max:42520 - global_seqlen/minmax_diff:10605 - global_seqlen/balanced_min:37284 - global_seqlen/balanced_max:37334 - global_seqlen/mean:37307.375 - actor/entropy:0.746766984462738 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48574504256248474 - training/rollout_probs_diff_mean:0.004121627192944288 - training/rollout_probs_diff_std:0.008096432313323021 - training/rollout_actor_probs_pearson_corr:0.999716579914093 - rollout_corr/rollout_is_mean:1.0002018213272095 - rollout_corr/rollout_is_ratio_fraction_high:4.438073119672481e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.438073119672481e-06 - rollout_corr/rollout_is_max:2.379199504852295 - rollout_corr/rollout_is_min:0.4145638346672058 - rollout_corr/rollout_is_std:0.03402938321232796 - rollout_corr/rollout_is_eff_sample_size:0.9988438161069871 - rollout_corr/rollout_is_seq_mean:1.000188946723938 - rollout_corr/rollout_is_seq_std:0.0011295655276626348 - rollout_corr/rollout_is_seq_max:1.0031288862228394 - rollout_corr/rollout_is_seq_min:0.9967604875564575 - rollout_corr/rollout_is_seq_max_deviation:0.0032395124435424805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.173908702097833) - rollout_corr/training_log_ppl:np.float64(0.7561200592317618) - rollout_corr/kl:np.float64(0.0004855001487325261) - rollout_corr/k3_kl:np.float64(0.0005892427249136745) - rollout_corr/rollout_ppl:np.float64(2.1727346065454185) - rollout_corr/rollout_log_ppl:np.float64(0.755634555243887) - rollout_corr/log_ppl_diff:np.float64(0.00048550398787483573) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009925910853780806) - rollout_corr/log_ppl_diff_max:np.float64(0.0035272836685180664) - rollout_corr/log_ppl_diff_min:np.float64(-0.002400696277618408) - rollout_corr/ppl_ratio:np.float64(1.0004863012582064) - rollout_corr/chi2_token:np.float64(0.0013905672822147608) - rollout_corr/chi2_seq:np.float64(0.9606744900811464) - actor/pg_loss:np.float64(-2.3183762095868587e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.338781410406227e-05) - actor/ppo_kl:np.float64(8.314622859639798e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18520821258425713) - perf/mfu/actor:np.float64(0.13935883033564134) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.53511428833008) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00035671013756655157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00035671013756655157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:880.16796875 - response_length/max:2522.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:880.16796875 - response_length_non_aborted/max:2522.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001479722559452057 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7096176706254482) - timing_s/agent_loop/generate_sequences/max:np.float64(20.600152047351003) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.33266173733864) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.600152047351003) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:196 - timing_s/agent_loop/slowest/response_length:2522 - timing_s/gen:27.267525793984532 - timing_s/reward:0.08974926918745041 - timing_s/old_log_prob:3.3883168008178473 - timing_s/adv:0.11706031486392021 - timing_s/update_actor:13.990974662825465 - timing_s/step:44.949168203398585 - timing_s/stop_profile:0.00010977871716022491 - timing_per_token_ms/adv:0.0003922157310180635 - timing_per_token_ms/gen:0.12101527937221027 - timing_per_token_ms/update_actor:0.0468773756624041 - perf/total_num_tokens:298459 - perf/time_per_step:44.949168203398585 - perf/throughput:829.990331104263 (TaskRunner pid=2224264) Training Progress: 76%|███████▌ | 76/100 [1:15:00<21:59, 54.96s/it] (TaskRunner pid=2224264) step:77 - global_seqlen/min:27604 - global_seqlen/max:39041 - global_seqlen/minmax_diff:11437 - global_seqlen/balanced_min:34288 - global_seqlen/balanced_max:34332 - global_seqlen/mean:34317.375 - actor/entropy:0.8058213591575623 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28673720359802246 - training/rollout_probs_diff_mean:0.004518347326666117 - training/rollout_probs_diff_std:0.008365931920707226 - training/rollout_actor_probs_pearson_corr:0.999616265296936 - rollout_corr/rollout_is_mean:0.9999557733535767 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6204079389572144 - rollout_corr/rollout_is_min:0.6416282057762146 - rollout_corr/rollout_is_std:0.03523077070713043 - rollout_corr/rollout_is_eff_sample_size:0.9987602126485932 - rollout_corr/rollout_is_seq_mean:0.9999356269836426 - rollout_corr/rollout_is_seq_std:0.0013528199633583426 - rollout_corr/rollout_is_seq_max:1.0050373077392578 - rollout_corr/rollout_is_seq_min:0.9947447180747986 - rollout_corr/rollout_is_seq_max_deviation:0.005255281925201416 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.28120540920645) - rollout_corr/training_log_ppl:np.float64(0.8050968096358702) - rollout_corr/kl:np.float64(0.0006265352302827765) - rollout_corr/k3_kl:np.float64(0.0006796217078317568) - rollout_corr/rollout_ppl:np.float64(2.279738489538431) - rollout_corr/rollout_log_ppl:np.float64(0.8044702711631544) - rollout_corr/log_ppl_diff:np.float64(0.0006265384727157652) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010923692607320845) - rollout_corr/log_ppl_diff_max:np.float64(0.005904078483581543) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028671622276306152) - rollout_corr/ppl_ratio:np.float64(1.0006276031490415) - rollout_corr/chi2_token:np.float64(0.0014646041672676802) - rollout_corr/chi2_seq:np.float64(0.7338430834934115) - actor/pg_loss:np.float64(0.00012833683285862207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017851591564976843) - actor/ppo_kl:np.float64(-6.952557651018765e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2424900233745575) - perf/mfu/actor:np.float64(0.13009512944939702) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.56500244140625) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007992738857865334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007992738857865334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:755.32421875 - response_length/max:2071.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:755.32421875 - response_length_non_aborted/max:2071.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013962015509605408 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1678621973842382) - timing_s/agent_loop/generate_sequences/max:np.float64(17.2276831548661) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.9278243403023225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.2276831548661) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:2071 - timing_s/gen:24.56353418342769 - timing_s/reward:0.08003837987780571 - timing_s/old_log_prob:3.337373360991478 - timing_s/adv:0.12115020304918289 - timing_s/update_actor:13.919541602954268 - timing_s/step:42.11578721739352 - timing_s/stop_profile:0.000111408531665802 - timing_per_token_ms/adv:0.00044128594862363047 - timing_per_token_ms/gen:0.12703326998147366 - timing_per_token_ms/update_actor:0.05070150908597419 - perf/total_num_tokens:274539 - perf/time_per_step:42.11578721739352 - perf/throughput:814.8339914165767 (TaskRunner pid=2224264) Training Progress: 77%|███████▋ | 77/100 [1:15:42<19:35, 51.12s/it] (TaskRunner pid=2224264) step:78 - global_seqlen/min:28938 - global_seqlen/max:37974 - global_seqlen/minmax_diff:9036 - global_seqlen/balanced_min:34229 - global_seqlen/balanced_max:34251 - global_seqlen/mean:34245.0 - actor/entropy:0.7469825148582458 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48943376541137695 - training/rollout_probs_diff_mean:0.004324706271290779 - training/rollout_probs_diff_std:0.008407223038375378 - training/rollout_actor_probs_pearson_corr:0.9996141195297241 - rollout_corr/rollout_is_mean:1.0000590085983276 - rollout_corr/rollout_is_ratio_fraction_high:4.9409068196837325e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.881813639367465e-06 - rollout_corr/rollout_is_max:4.681254863739014 - rollout_corr/rollout_is_min:0.4222910404205322 - rollout_corr/rollout_is_std:0.03437097743153572 - rollout_corr/rollout_is_eff_sample_size:0.9988201488429446 - rollout_corr/rollout_is_seq_mean:1.00004243850708 - rollout_corr/rollout_is_seq_std:0.001362150302156806 - rollout_corr/rollout_is_seq_max:1.0039869546890259 - rollout_corr/rollout_is_seq_min:0.9943282008171082 - rollout_corr/rollout_is_seq_max_deviation:0.005671799182891846 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1253380407579243) - rollout_corr/training_log_ppl:np.float64(0.7371736962813884) - rollout_corr/kl:np.float64(0.0005827662733111083) - rollout_corr/k3_kl:np.float64(0.0006460418212554941) - rollout_corr/rollout_ppl:np.float64(2.1241166265681386) - rollout_corr/rollout_log_ppl:np.float64(0.7365909329382703) - rollout_corr/log_ppl_diff:np.float64(0.0005827633431181312) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012038786662742496) - rollout_corr/log_ppl_diff_max:np.float64(0.006325364112854004) - rollout_corr/log_ppl_diff_min:np.float64(-0.003412604331970215) - rollout_corr/ppl_ratio:np.float64(1.0005839837249368) - rollout_corr/chi2_token:np.float64(0.0014792552683502436) - rollout_corr/chi2_seq:np.float64(2.1139756981283426) - actor/pg_loss:np.float64(0.0001295657129958272) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001085545256955811) - actor/ppo_kl:np.float64(1.3940017076219391e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20791660994291306) - perf/mfu/actor:np.float64(0.1296498704894837) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.49341583251953) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018497519195079803 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018497519195079803 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:790.59375 - response_length/max:1840.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:790.59375 - response_length_non_aborted/max:1840.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010417215526103973 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2816403061151505) - timing_s/agent_loop/generate_sequences/max:np.float64(16.65826540067792) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.189224266308884) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.65826540067792) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:1840 - timing_s/gen:23.61698125116527 - timing_s/reward:0.08289889059960842 - timing_s/old_log_prob:3.274001955986023 - timing_s/adv:0.12328475899994373 - timing_s/update_actor:13.82346528582275 - timing_s/step:41.00753286853433 - timing_s/stop_profile:0.00011822767555713654 - timing_per_token_ms/adv:0.000450010070813052 - timing_per_token_ms/gen:0.11668930220149645 - timing_per_token_ms/update_actor:0.050457969359843584 - perf/total_num_tokens:273960 - perf/time_per_step:41.00753286853433 - perf/throughput:835.0904725184452 (TaskRunner pid=2224264) Training Progress: 78%|███████▊ | 78/100 [1:16:23<17:38, 48.10s/it] (TaskRunner pid=2224264) step:79 - global_seqlen/min:31052 - global_seqlen/max:40489 - global_seqlen/minmax_diff:9437 - global_seqlen/balanced_min:35267 - global_seqlen/balanced_max:35291 - global_seqlen/mean:35282.375 - actor/entropy:0.7419886589050293 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3267946243286133 - training/rollout_probs_diff_mean:0.004350726492702961 - training/rollout_probs_diff_std:0.008377747610211372 - training/rollout_actor_probs_pearson_corr:0.9996976256370544 - rollout_corr/rollout_is_mean:1.0000663995742798 - rollout_corr/rollout_is_ratio_fraction_high:4.6575100896006916e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.315020179201383e-06 - rollout_corr/rollout_is_max:2.486323833465576 - rollout_corr/rollout_is_min:0.3386184871196747 - rollout_corr/rollout_is_std:0.034230221062898636 - rollout_corr/rollout_is_eff_sample_size:0.9988299010470403 - rollout_corr/rollout_is_seq_mean:1.0000535249710083 - rollout_corr/rollout_is_seq_std:0.001242220401763916 - rollout_corr/rollout_is_seq_max:1.0033190250396729 - rollout_corr/rollout_is_seq_min:0.9950276017189026 - rollout_corr/rollout_is_seq_max_deviation:0.004972398281097412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1713191107846797) - rollout_corr/training_log_ppl:np.float64(0.7553618648089468) - rollout_corr/kl:np.float64(0.0005726922157927028) - rollout_corr/k3_kl:np.float64(0.0006373734220233018) - rollout_corr/rollout_ppl:np.float64(2.170036384370178) - rollout_corr/rollout_log_ppl:np.float64(0.754789168946445) - rollout_corr/log_ppl_diff:np.float64(0.0005726958625018597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010784545447677374) - rollout_corr/log_ppl_diff_max:np.float64(0.004006564617156982) - rollout_corr/log_ppl_diff_min:np.float64(-0.00302046537399292) - rollout_corr/ppl_ratio:np.float64(1.0005736567545682) - rollout_corr/chi2_token:np.float64(0.001405506394803524) - rollout_corr/chi2_seq:np.float64(1.6988238336052746) - actor/pg_loss:np.float64(6.845139432698488e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017011942372846534) - actor/ppo_kl:np.float64(1.6240256268673647e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24053851515054703) - perf/mfu/actor:np.float64(0.13349716649984694) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.54805374145508) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011310762725770473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011310762725770473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:838.69921875 - response_length/max:2007.0 - response_length/min:283.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:838.69921875 - response_length_non_aborted/max:2007.0 - response_length_non_aborted/min:283.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010603666305541992 - timing_s/agent_loop/generate_sequences/min:np.float64(3.655642168596387) - timing_s/agent_loop/generate_sequences/max:np.float64(17.38879804685712) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.424919297925953) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.38879804685712) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:2007 - timing_s/gen:24.167077971622348 - timing_s/reward:0.08212696015834808 - timing_s/old_log_prob:3.3000302650034428 - timing_s/adv:0.11685498803853989 - timing_s/update_actor:13.943561296910048 - timing_s/step:41.69688728451729 - timing_s/stop_profile:0.00011464767158031464 - timing_per_token_ms/adv:0.00041399915693933546 - timing_per_token_ms/gen:0.11255840737201092 - timing_per_token_ms/update_actor:0.04939988201230093 - perf/total_num_tokens:282259 - perf/time_per_step:41.69688728451729 - perf/throughput:846.1632821474639 (TaskRunner pid=2224264) Training Progress: 79%|███████▉ | 79/100 [1:17:05<16:10, 46.19s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a target product: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) This is a nitrogen-containing compound with a ring structure. Let's analyze it: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - The formula appears to represent a **pyrrole ring** (a 5-membered ring with one nitrogen atom). (TaskRunner pid=2224264) - The nitrogen is at the first position (Nc1...), and there's a substituent at the third carbon: **Cc2ccc(Cl)cc2C(F)(F)F**, which implies a **benzene ring** with a **chlorine atom** and a **trifluoromethyl group (C(F)(F)F)** on the same carbon. (TaskRunner pid=2224264) - So the full structure is: a **pyrrole ring** with a **benzene ring** substituted with **Cl** and **CF₃** at the same position, attached to the third carbon of the pyrrole. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now we examine the options to find which one is the **correct reactant** that would lead to this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) - This has a **pyrrole ring** (from the N1 and the 5-membered ring). (TaskRunner pid=2224264) - The ring is substituted with a **benzene ring** (c2ccccc2) with a **Cl** and **CF₃** at the same position. (TaskRunner pid=2224264) - The structure matches the **target product** exactly. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) - This contains a **Cl** on the ring, but the ring structure is different (not a pyrrole, and the Cl is not in the correct position). (TaskRunner pid=2224264) - The structure is not matching the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and CF₃, but it's attached to an **oxygen** (Oc2...), not directly to the pyrrole. (TaskRunner pid=2224264) - The structure is different from the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) - This has a **trifluoromethyl group** (C(F)(F)F), but it is attached to a **methylene group** (CC=C) and not directly to the ring. (TaskRunner pid=2224264) - The structure is not matching the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Conclusion: (TaskRunner pid=2224264) Only **Option A** has the correct **pyrrole ring** with the correct **benzene substituent** (Cl and CF₃) in the correct position, matching the structure of the given product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2224264) step:80 - global_seqlen/min:28862 - global_seqlen/max:47477 - global_seqlen/minmax_diff:18615 - global_seqlen/balanced_min:36267 - global_seqlen/balanced_max:36835 - global_seqlen/mean:36375.375 - actor/entropy:0.7752931714057922 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967353940010071 - training/rollout_probs_diff_mean:0.004443757236003876 - training/rollout_probs_diff_std:0.008464205078780651 - training/rollout_actor_probs_pearson_corr:0.9995999336242676 - rollout_corr/rollout_is_mean:1.0000269412994385 - rollout_corr/rollout_is_ratio_fraction_high:9.492214303463697e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.492214303463697e-06 - rollout_corr/rollout_is_max:2.5489871501922607 - rollout_corr/rollout_is_min:0.2599738538265228 - rollout_corr/rollout_is_std:0.03524937480688095 - rollout_corr/rollout_is_eff_sample_size:0.99875914242478 - rollout_corr/rollout_is_seq_mean:0.9999639391899109 - rollout_corr/rollout_is_seq_std:0.0014692033873870969 - rollout_corr/rollout_is_seq_max:1.0044381618499756 - rollout_corr/rollout_is_seq_min:0.9949309825897217 - rollout_corr/rollout_is_seq_max_deviation:0.00506901741027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2062102765776217) - rollout_corr/training_log_ppl:np.float64(0.7724762546131387) - rollout_corr/kl:np.float64(0.0008447698922706692) - rollout_corr/k3_kl:np.float64(0.0006712018573580281) - rollout_corr/rollout_ppl:np.float64(2.204339082352817) - rollout_corr/rollout_log_ppl:np.float64(0.7716314849676564) - rollout_corr/log_ppl_diff:np.float64(0.0008447696454823017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013305179309099913) - rollout_corr/log_ppl_diff_max:np.float64(0.005142807960510254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035692453384399414) - rollout_corr/ppl_ratio:np.float64(1.0008462956175208) - rollout_corr/chi2_token:np.float64(0.0010051389690488577) - rollout_corr/chi2_seq:np.float64(3.9995412554126233) - actor/pg_loss:np.float64(6.170920096337795e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.559897284816543e-05) - actor/ppo_kl:np.float64(0.00016689712089679176) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20688721537590027) - perf/mfu/actor:np.float64(0.1360888320089338) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.50061416625977) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6782738095238096) - val-aux/sciknoweval/reward/std@16:np.float64(0.2194428960706272) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7662047619047618) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1707809803907824) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5911047619047619) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.19703442080325248) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6786285714285715) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2187494224175815) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8316666666666666) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1211692975832172) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.513647619047619) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16233574997526817) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7021999999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17154812598360286) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8789666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08054198286562807) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44790952380952387) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12433382532071413) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7142047619047619) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12568969808131827) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9095333333333334) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0468538841472672) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3950142857142857) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08487101067852891) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7184190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09186432979311497) - val-aux/sciknoweval/score/mean@16:np.float64(0.6782738095238096) - val-aux/sciknoweval/score/std@16:np.float64(0.2194428960706272) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7662047619047618) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1707809803907824) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5911047619047619) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.19703442080325248) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6786285714285715) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2187494224175815) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8316666666666666) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1211692975832172) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.513647619047619) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16233574997526817) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7021999999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17154812598360286) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8789666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08054198286562807) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.44790952380952387) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12433382532071413) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7142047619047619) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12568969808131827) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9095333333333334) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0468538841472672) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3950142857142857) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08487101067852891) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7184190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09186432979311497) - val-core/sciknoweval/acc/mean@16:np.float64(0.6782738095238096) - val-aux/sciknoweval/acc/std@16:np.float64(0.2194428960706272) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7662047619047618) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1707809803907824) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5911047619047619) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.19703442080325248) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6786285714285715) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2187494224175815) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8316666666666666) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1211692975832172) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.513647619047619) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16233574997526817) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7021999999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17154812598360286) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8789666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08054198286562807) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.44790952380952387) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12433382532071413) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7142047619047619) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12568969808131827) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9095333333333334) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0468538841472672) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3950142857142857) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08487101067852891) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7184190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09186432979311497) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9797619047619047) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0529405396018874) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9968523809523809) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02007926101605294) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9632904761904761) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06706255313069127) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9803428571428572) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.052420251171557214) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998619047619047) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.002377576095584018) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9359809523809526) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07827091020757276) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9914190476190478) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.034868422157064094) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8997619047619049) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07810157852659988) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9974428571428571) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.017502498474107602) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.863804761904762) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06109163301171088) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996809523809523) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004826434845870424) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007080005016177893 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007080005016177893 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:823.04296875 - response_length/max:3037.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:823.04296875 - response_length_non_aborted/max:3037.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014653801918029785 - timing_s/agent_loop/generate_sequences/min:np.float64(2.194973863661289) - timing_s/agent_loop/generate_sequences/max:np.float64(22.72264883853495) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.372557892595069) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.72264883853495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:3037 - timing_s/gen:30.101325387135148 - timing_s/reward:0.08319770731031895 - timing_s/old_log_prob:3.4729851465672255 - timing_s/adv:0.14867497608065605 - timing_s/update_actor:14.160021048039198 - timing_s/step:48.05162427574396 - timing_s/testing:207.67929325439036 - timing_s/save_checkpoint:12.710731318220496 - timing_s/stop_profile:0.00015132129192352295 - timing_per_token_ms/adv:0.0005109053036589178 - timing_per_token_ms/gen:0.14286411130159682 - timing_per_token_ms/update_actor:0.04865936450153159 - perf/total_num_tokens:291003 - perf/time_per_step:48.05162427574396 - perf/throughput:757.0061480390366 (TaskRunner pid=2224264) Training Progress: 80%|████████ | 80/100 [1:21:33<37:37, 112.89s/it] (TaskRunner pid=2224264) step:81 - global_seqlen/min:23435 - global_seqlen/max:40414 - global_seqlen/minmax_diff:16979 - global_seqlen/balanced_min:31718 - global_seqlen/balanced_max:31758 - global_seqlen/mean:31742.875 - actor/entropy:0.728011965751648 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25490427017211914 - training/rollout_probs_diff_mean:0.00432547414675355 - training/rollout_probs_diff_std:0.00843879021704197 - training/rollout_actor_probs_pearson_corr:0.9996644258499146 - rollout_corr/rollout_is_mean:0.9999246001243591 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7127294995589182e-05 - rollout_corr/rollout_is_max:1.6446915864944458 - rollout_corr/rollout_is_min:0.3955734968185425 - rollout_corr/rollout_is_std:0.03461290895938873 - rollout_corr/rollout_is_eff_sample_size:0.9988031424060505 - rollout_corr/rollout_is_seq_mean:0.9999666810035706 - rollout_corr/rollout_is_seq_std:0.001528046908788383 - rollout_corr/rollout_is_seq_max:1.0054874420166016 - rollout_corr/rollout_is_seq_min:0.9947968125343323 - rollout_corr/rollout_is_seq_max_deviation:0.0054874420166015625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.084522245451808) - rollout_corr/training_log_ppl:np.float64(0.7124685312155634) - rollout_corr/kl:np.float64(0.0007158686560133631) - rollout_corr/k3_kl:np.float64(0.000656772480510881) - rollout_corr/rollout_ppl:np.float64(2.083047821652144) - rollout_corr/rollout_log_ppl:np.float64(0.7117526578949764) - rollout_corr/log_ppl_diff:np.float64(0.0007158733205869794) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012598995817825198) - rollout_corr/log_ppl_diff_max:np.float64(0.005965232849121094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034725069999694824) - rollout_corr/ppl_ratio:np.float64(1.000717243179679) - rollout_corr/chi2_token:np.float64(0.0012002624571323395) - rollout_corr/chi2_seq:np.float64(0.9090896914713085) - actor/pg_loss:np.float64(-3.0948780477046967e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011802908329627826) - actor/ppo_kl:np.float64(6.148918862525221e-05) - actor/pg_clipfrac_lower:np.float64(2.239822151750559e-06) - actor/grad_norm:np.float64(0.2090231031179428) - perf/mfu/actor:np.float64(0.12132654821351155) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.81432342529297) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020438572391867638 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020438572391867638 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:684.21484375 - response_length/max:1807.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:684.21484375 - response_length_non_aborted/max:1807.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001064930111169815 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4930436480790377) - timing_s/agent_loop/generate_sequences/max:np.float64(14.782667718827724) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.092499443388078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.782667718827724) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1744 - timing_s/gen:21.074421510100365 - timing_s/reward:0.071615781635046 - timing_s/old_log_prob:3.5168352890759706 - timing_s/adv:0.12568366713821888 - timing_s/update_actor:13.775931548327208 - timing_s/step:38.66653628088534 - timing_s/stop_profile:0.00012343190610408783 - timing_per_token_ms/adv:0.0004949286538247516 - timing_per_token_ms/gen:0.12031595013730591 - timing_per_token_ms/update_actor:0.05424812476944514 - perf/total_num_tokens:253943 - perf/time_per_step:38.66653628088534 - perf/throughput:820.9391906585637 (TaskRunner pid=2224264) Training Progress: 81%|████████ | 81/100 [1:22:12<28:42, 90.64s/it] (TaskRunner pid=2224264) step:82 - global_seqlen/min:30553 - global_seqlen/max:40093 - global_seqlen/minmax_diff:9540 - global_seqlen/balanced_min:34588 - global_seqlen/balanced_max:34791 - global_seqlen/mean:34631.0 - actor/entropy:0.7073292136192322 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48784658312797546 - training/rollout_probs_diff_mean:0.004207909107208252 - training/rollout_probs_diff_std:0.008302947506308556 - training/rollout_actor_probs_pearson_corr:0.9996032118797302 - rollout_corr/rollout_is_mean:0.9999532103538513 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4358463886310346e-05 - rollout_corr/rollout_is_max:1.861779808998108 - rollout_corr/rollout_is_min:0.3777696490287781 - rollout_corr/rollout_is_std:0.033776216208934784 - rollout_corr/rollout_is_eff_sample_size:0.998860348177861 - rollout_corr/rollout_is_seq_mean:0.9999899864196777 - rollout_corr/rollout_is_seq_std:0.0011922791600227356 - rollout_corr/rollout_is_seq_max:1.004441738128662 - rollout_corr/rollout_is_seq_min:0.9973618984222412 - rollout_corr/rollout_is_seq_max_deviation:0.004441738128662109 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0682692220434546) - rollout_corr/training_log_ppl:np.float64(0.7130721984431148) - rollout_corr/kl:np.float64(0.0005954480367265091) - rollout_corr/k3_kl:np.float64(0.0006088201273541927) - rollout_corr/rollout_ppl:np.float64(2.067028284072876) - rollout_corr/rollout_log_ppl:np.float64(0.712476747808978) - rollout_corr/log_ppl_diff:np.float64(0.000595450634136796) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011131800711154938) - rollout_corr/log_ppl_diff_max:np.float64(0.0041391849517822266) - rollout_corr/log_ppl_diff_min:np.float64(-0.00474703311920166) - rollout_corr/ppl_ratio:np.float64(1.0005964431911707) - rollout_corr/chi2_token:np.float64(0.0012475282419472933) - rollout_corr/chi2_seq:np.float64(2.9451562664471567) - actor/pg_loss:np.float64(0.00018601585179567337) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019039797803088732) - actor/ppo_kl:np.float64(-5.924717907390686e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25703465938568115) - perf/mfu/actor:np.float64(0.1307738033068337) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.3072280883789) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006862149457447231 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006862149457447231 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:816.15625 - response_length/max:2571.0 - response_length/min:362.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:816.15625 - response_length_non_aborted/max:2571.0 - response_length_non_aborted/min:362.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.565187454223633e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(4.610755139961839) - timing_s/agent_loop/generate_sequences/max:np.float64(20.74920883215964) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.369914709837758) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.74920883215964) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:2571 - timing_s/gen:27.471973603591323 - timing_s/reward:0.07828585989773273 - timing_s/old_log_prob:3.2765041179955006 - timing_s/adv:0.11844107136130333 - timing_s/update_actor:13.957106951624155 - timing_s/step:44.991200095042586 - timing_s/stop_profile:0.00011249631643295288 - timing_per_token_ms/adv:0.0004275110138362426 - timing_per_token_ms/gen:0.13148511316188366 - timing_per_token_ms/update_actor:0.05037793794441452 - perf/total_num_tokens:277048 - perf/time_per_step:44.991200095042586 - perf/throughput:769.7283007975567 (TaskRunner pid=2224264) Training Progress: 82%|████████▏ | 82/100 [1:22:57<23:05, 76.96s/it] (TaskRunner pid=2224264) step:83 - global_seqlen/min:24830 - global_seqlen/max:36777 - global_seqlen/minmax_diff:11947 - global_seqlen/balanced_min:31744 - global_seqlen/balanced_max:31974 - global_seqlen/mean:31783.0 - actor/entropy:0.7531368136405945 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977966308594 - training/rollout_probs_diff_mean:0.0044290670193731785 - training/rollout_probs_diff_std:0.008538827300071716 - training/rollout_actor_probs_pearson_corr:0.999602735042572 - rollout_corr/rollout_is_mean:1.0000147819519043 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6260687415488064e-06 - rollout_corr/rollout_is_max:1.5903000831604004 - rollout_corr/rollout_is_min:0.4172116219997406 - rollout_corr/rollout_is_std:0.03477611765265465 - rollout_corr/rollout_is_eff_sample_size:0.9987920825865458 - rollout_corr/rollout_is_seq_mean:0.9999656677246094 - rollout_corr/rollout_is_seq_std:0.0014977967366576195 - rollout_corr/rollout_is_seq_max:1.0056312084197998 - rollout_corr/rollout_is_seq_min:0.9933651685714722 - rollout_corr/rollout_is_seq_max_deviation:0.006634831428527832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.162785124965012) - rollout_corr/training_log_ppl:np.float64(0.7481051302747801) - rollout_corr/kl:np.float64(0.0006437221610795518) - rollout_corr/k3_kl:np.float64(0.0006854262796878174) - rollout_corr/rollout_ppl:np.float64(2.161293275654316) - rollout_corr/rollout_log_ppl:np.float64(0.7474614095408469) - rollout_corr/log_ppl_diff:np.float64(0.0006437207339331508) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012794457143172622) - rollout_corr/log_ppl_diff_max:np.float64(0.0074920654296875) - rollout_corr/log_ppl_diff_min:np.float64(-0.004778653383255005) - rollout_corr/ppl_ratio:np.float64(1.000645110849291) - rollout_corr/chi2_token:np.float64(0.0014334383886307478) - rollout_corr/chi2_seq:np.float64(0.4927413531113416) - actor/pg_loss:np.float64(-0.00020378001499921083) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024108612069539959) - actor/ppo_kl:np.float64(-1.7973163554674265e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23665034770965576) - perf/mfu/actor:np.float64(0.12241472211925314) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.91914367675781) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008029807358980179 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008029807358980179 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:694.3125 - response_length/max:2176.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:694.3125 - response_length_non_aborted/max:2176.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010167807340621948 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2234518099576235) - timing_s/agent_loop/generate_sequences/max:np.float64(17.420780858024955) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.779617345870065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.420780858024955) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:2176 - timing_s/gen:23.951165637001395 - timing_s/reward:0.07695263624191284 - timing_s/old_log_prob:3.1909585148096085 - timing_s/adv:0.11885049380362034 - timing_s/update_actor:13.669648755341768 - timing_s/step:41.09482858143747 - timing_s/stop_profile:0.00010753236711025238 - timing_per_token_ms/adv:0.00046742949770168146 - timing_per_token_ms/gen:0.13475090938091522 - timing_per_token_ms/update_actor:0.05376163654839761 - perf/total_num_tokens:254264 - perf/time_per_step:41.09482858143747 - perf/throughput:773.4063164910336 (TaskRunner pid=2224264) Training Progress: 83%|████████▎ | 83/100 [1:23:38<18:45, 66.22s/it] (TaskRunner pid=2224264) step:84 - global_seqlen/min:20654 - global_seqlen/max:34303 - global_seqlen/minmax_diff:13649 - global_seqlen/balanced_min:29574 - global_seqlen/balanced_max:29701 - global_seqlen/mean:29614.0 - actor/entropy:0.6809946894645691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43606680631637573 - training/rollout_probs_diff_mean:0.004484572447836399 - training/rollout_probs_diff_std:0.008784463629126549 - training/rollout_actor_probs_pearson_corr:0.999607264995575 - rollout_corr/rollout_is_mean:0.9998617172241211 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1636025192274246e-05 - rollout_corr/rollout_is_max:1.5042952299118042 - rollout_corr/rollout_is_min:0.4855743646621704 - rollout_corr/rollout_is_std:0.0344783253967762 - rollout_corr/rollout_is_eff_sample_size:0.9988122996462617 - rollout_corr/rollout_is_seq_mean:0.9998700618743896 - rollout_corr/rollout_is_seq_std:0.0014390589203685522 - rollout_corr/rollout_is_seq_max:1.0044506788253784 - rollout_corr/rollout_is_seq_min:0.9937525391578674 - rollout_corr/rollout_is_seq_max_deviation:0.006247460842132568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9907695450820029) - rollout_corr/training_log_ppl:np.float64(0.670324468635954) - rollout_corr/kl:np.float64(0.0006556954003826831) - rollout_corr/k3_kl:np.float64(0.000665820000790518) - rollout_corr/rollout_ppl:np.float64(1.9894441780634224) - rollout_corr/rollout_log_ppl:np.float64(0.6696687689982355) - rollout_corr/log_ppl_diff:np.float64(0.0006556996377184987) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012462410377338529) - rollout_corr/log_ppl_diff_max:np.float64(0.007571697235107422) - rollout_corr/log_ppl_diff_min:np.float64(-0.004120081663131714) - rollout_corr/ppl_ratio:np.float64(1.000657090684399) - rollout_corr/chi2_token:np.float64(0.0013479453045874834) - rollout_corr/chi2_seq:np.float64(0.807735110167414) - actor/pg_loss:np.float64(-0.00014557677786797285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001528950670035556) - actor/ppo_kl:np.float64(-9.21530613155197e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18504177778959274) - perf/mfu/actor:np.float64(0.1154202224364208) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.7387924194336) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01133712474256754 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01133712474256754 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:671.40625 - response_length/max:2226.0 - response_length/min:177.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:671.40625 - response_length_non_aborted/max:2226.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017830543220043182 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6220184583216906) - timing_s/agent_loop/generate_sequences/max:np.float64(16.999597314745188) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.399542242870666) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.999597314745188) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:177 - timing_s/agent_loop/slowest/response_length:2226 - timing_s/gen:24.433550149202347 - timing_s/reward:0.07261811941862106 - timing_s/old_log_prob:3.231289431452751 - timing_s/adv:0.12676293589174747 - timing_s/update_actor:13.54585151001811 - timing_s/step:41.49533977732062 - timing_s/stop_profile:0.00012038461863994598 - timing_per_token_ms/adv:0.0005350633817271707 - timing_per_token_ms/gen:0.14215470182221518 - timing_per_token_ms/update_actor:0.05717672177862713 - perf/total_num_tokens:236912 - perf/time_per_step:41.49533977732062 - perf/throughput:713.6705027340348 (TaskRunner pid=2224264) Training Progress: 84%|████████▍ | 84/100 [1:24:20<15:41, 58.82s/it] (TaskRunner pid=2224264) step:85 - global_seqlen/min:27367 - global_seqlen/max:37161 - global_seqlen/minmax_diff:9794 - global_seqlen/balanced_min:32947 - global_seqlen/balanced_max:32956 - global_seqlen/mean:32953.5 - actor/entropy:0.735228419303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948740243911743 - training/rollout_probs_diff_mean:0.004468359053134918 - training/rollout_probs_diff_std:0.008735636249184608 - training/rollout_actor_probs_pearson_corr:0.9996408224105835 - rollout_corr/rollout_is_mean:0.9999686479568481 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0702284271246754e-05 - rollout_corr/rollout_is_max:1.7922857999801636 - rollout_corr/rollout_is_min:0.23469160497188568 - rollout_corr/rollout_is_std:0.03531694784760475 - rollout_corr/rollout_is_eff_sample_size:0.9987541480773114 - rollout_corr/rollout_is_seq_mean:1.0000033378601074 - rollout_corr/rollout_is_seq_std:0.0015298437792807817 - rollout_corr/rollout_is_seq_max:1.0047695636749268 - rollout_corr/rollout_is_seq_min:0.9963509440422058 - rollout_corr/rollout_is_seq_max_deviation:0.004769563674926758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1323386910371482) - rollout_corr/training_log_ppl:np.float64(0.7435957858106121) - rollout_corr/kl:np.float64(0.0005658202565279602) - rollout_corr/k3_kl:np.float64(0.0006766391076098444) - rollout_corr/rollout_ppl:np.float64(2.1311398232355714) - rollout_corr/rollout_log_ppl:np.float64(0.7430299640400335) - rollout_corr/log_ppl_diff:np.float64(0.0005658217705786228) - rollout_corr/log_ppl_abs_diff:np.float64(0.00123508065007627) - rollout_corr/log_ppl_diff_max:np.float64(0.004446566104888916) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032745003700256348) - rollout_corr/ppl_ratio:np.float64(1.000567066250369) - rollout_corr/chi2_token:np.float64(0.0015738557558506727) - rollout_corr/chi2_seq:np.float64(1.2473218482919037) - actor/pg_loss:np.float64(-0.00010735273826867342) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.374866226607992e-05) - actor/ppo_kl:np.float64(-8.093407468590108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18677664920687675) - perf/mfu/actor:np.float64(0.1273840230570504) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.69750595092773) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00654712226241827 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00654712226241827 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:729.984375 - response_length/max:1647.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:729.984375 - response_length_non_aborted/max:1647.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014764443039894104 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8548108004033566) - timing_s/agent_loop/generate_sequences/max:np.float64(14.612083107233047) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.586479216020962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.612083107233047) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:205 - timing_s/agent_loop/slowest/response_length:1647 - timing_s/gen:21.41356766782701 - timing_s/reward:0.08048001676797867 - timing_s/old_log_prob:3.2531582564115524 - timing_s/adv:0.11875008791685104 - timing_s/update_actor:13.537027467042208 - timing_s/step:38.487668914720416 - timing_s/stop_profile:0.00011523440480232239 - timing_per_token_ms/adv:0.00045044565796065303 - timing_per_token_ms/gen:0.11458703989718856 - timing_per_token_ms/update_actor:0.051348974566594624 - perf/total_num_tokens:263628 - perf/time_per_step:38.487668914720416 - perf/throughput:856.2092984383432 (TaskRunner pid=2224264) Training Progress: 85%|████████▌ | 85/100 [1:24:58<13:11, 52.74s/it] (TaskRunner pid=2224264) step:86 - global_seqlen/min:24044 - global_seqlen/max:35362 - global_seqlen/minmax_diff:11318 - global_seqlen/balanced_min:30989 - global_seqlen/balanced_max:31001 - global_seqlen/mean:30998.375 - actor/entropy:0.6628086566925049 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4241427183151245 - training/rollout_probs_diff_mean:0.004253617953509092 - training/rollout_probs_diff_std:0.008793003857135773 - training/rollout_actor_probs_pearson_corr:0.9995636940002441 - rollout_corr/rollout_is_mean:1.00003182888031 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.650387720379513e-06 - rollout_corr/rollout_is_max:1.8955367803573608 - rollout_corr/rollout_is_min:0.47840172052383423 - rollout_corr/rollout_is_std:0.03403814136981964 - rollout_corr/rollout_is_eff_sample_size:0.998842864637548 - rollout_corr/rollout_is_seq_mean:0.999988317489624 - rollout_corr/rollout_is_seq_std:0.001431034179404378 - rollout_corr/rollout_is_seq_max:1.0039067268371582 - rollout_corr/rollout_is_seq_min:0.9941695332527161 - rollout_corr/rollout_is_seq_max_deviation:0.0058304667472839355 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.980902411043644) - rollout_corr/training_log_ppl:np.float64(0.6618329326156527) - rollout_corr/kl:np.float64(0.0007042251129403265) - rollout_corr/k3_kl:np.float64(0.0006341270048437764) - rollout_corr/rollout_ppl:np.float64(1.979446857701987) - rollout_corr/rollout_log_ppl:np.float64(0.6611287064151838) - rollout_corr/log_ppl_diff:np.float64(0.0007042262004688382) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012508378131315112) - rollout_corr/log_ppl_diff_max:np.float64(0.006446421146392822) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046353936195373535) - rollout_corr/ppl_ratio:np.float64(1.0007055627647787) - rollout_corr/chi2_token:np.float64(0.0011301424819976091) - rollout_corr/chi2_seq:np.float64(0.7622500413563102) - actor/pg_loss:np.float64(0.00021916942205280066) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026571475882519735) - actor/ppo_kl:np.float64(8.79584136299627e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2480062022805214) - perf/mfu/actor:np.float64(0.12020480007027705) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.7301025390625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002775753091555089 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002775753091555089 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:691.32421875 - response_length/max:1574.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:691.32421875 - response_length_non_aborted/max:1574.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017972663044929504 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9018322117626667) - timing_s/agent_loop/generate_sequences/max:np.float64(14.014412458986044) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.677346075157402) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.014412458986044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1521 - timing_s/gen:20.71387554332614 - timing_s/reward:0.0748683363199234 - timing_s/old_log_prob:3.207970952615142 - timing_s/adv:0.11068613082170486 - timing_s/update_actor:13.604356173425913 - timing_s/step:37.79681725241244 - timing_s/stop_profile:2.8118491172790527e-05 - timing_per_token_ms/adv:0.00044633844040899265 - timing_per_token_ms/gen:0.11704143171407987 - timing_per_token_ms/update_actor:0.05485915057412652 - perf/total_num_tokens:247987 - perf/time_per_step:37.79681725241244 - perf/throughput:820.1318855232839 (TaskRunner pid=2224264) Training Progress: 86%|████████▌ | 86/100 [1:25:36<11:15, 48.26s/it] (TaskRunner pid=2224264) step:87 - global_seqlen/min:27376 - global_seqlen/max:34559 - global_seqlen/minmax_diff:7183 - global_seqlen/balanced_min:30988 - global_seqlen/balanced_max:31014 - global_seqlen/mean:31004.375 - actor/entropy:0.6844121217727661 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45646268129348755 - training/rollout_probs_diff_mean:0.004233994521200657 - training/rollout_probs_diff_std:0.008540390990674496 - training/rollout_actor_probs_pearson_corr:0.9996417760848999 - rollout_corr/rollout_is_mean:1.000025749206543 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1601264304772485e-05 - rollout_corr/rollout_is_max:1.5985411405563354 - rollout_corr/rollout_is_min:0.412740021944046 - rollout_corr/rollout_is_std:0.033871378749608994 - rollout_corr/rollout_is_eff_sample_size:0.998854282390456 - rollout_corr/rollout_is_seq_mean:1.0000381469726562 - rollout_corr/rollout_is_seq_std:0.0015245388494804502 - rollout_corr/rollout_is_seq_max:1.0053566694259644 - rollout_corr/rollout_is_seq_min:0.9960179328918457 - rollout_corr/rollout_is_seq_max_deviation:0.0053566694259643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0139922401867807) - rollout_corr/training_log_ppl:np.float64(0.6768635550397448) - rollout_corr/kl:np.float64(0.0006538237156501481) - rollout_corr/k3_kl:np.float64(0.0006354092881792894) - rollout_corr/rollout_ppl:np.float64(2.0125835882499814) - rollout_corr/rollout_log_ppl:np.float64(0.6762097296887077) - rollout_corr/log_ppl_diff:np.float64(0.0006538253510370851) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013526437105610967) - rollout_corr/log_ppl_diff_max:np.float64(0.00598597526550293) - rollout_corr/log_ppl_diff_min:np.float64(-0.005899667739868164) - rollout_corr/ppl_ratio:np.float64(1.0006554168649018) - rollout_corr/chi2_token:np.float64(0.0012414318043738604) - rollout_corr/chi2_seq:np.float64(0.7429793914780021) - actor/pg_loss:np.float64(-8.32361401990056e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030786197271481797) - actor/ppo_kl:np.float64(9.98133720919725e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2665778249502182) - perf/mfu/actor:np.float64(0.12025084389863877) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.70088958740234) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005423591006547213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005423591006547213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:673.41796875 - response_length/max:1719.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:673.41796875 - response_length_non_aborted/max:1719.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.614036202430725e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9507676176726818) - timing_s/agent_loop/generate_sequences/max:np.float64(14.615875907242298) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.613317727045796) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.615875907242298) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1719 - timing_s/gen:21.28308396600187 - timing_s/reward:0.0755326934158802 - timing_s/old_log_prob:3.1504952367395163 - timing_s/adv:0.11150607839226723 - timing_s/update_actor:13.641991836950183 - timing_s/step:38.34868148341775 - timing_s/stop_profile:3.3913180232048035e-05 - timing_per_token_ms/adv:0.00044955783817714125 - timing_per_token_ms/gen:0.12345534363526708 - timing_per_token_ms/update_actor:0.055000269465801936 - perf/total_num_tokens:248035 - perf/time_per_step:38.34868148341775 - perf/throughput:808.4860756791995 (TaskRunner pid=2224264) Training Progress: 87%|████████▋ | 87/100 [1:26:14<09:48, 45.29s/it] (TaskRunner pid=2224264) step:88 - global_seqlen/min:25663 - global_seqlen/max:37476 - global_seqlen/minmax_diff:11813 - global_seqlen/balanced_min:30723 - global_seqlen/balanced_max:30761 - global_seqlen/mean:30750.875 - actor/entropy:0.6911125183105469 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.729590892791748 - training/rollout_probs_diff_mean:0.004324982408434153 - training/rollout_probs_diff_std:0.0088273286819458 - training/rollout_actor_probs_pearson_corr:0.9995751976966858 - rollout_corr/rollout_is_mean:0.9999687671661377 - rollout_corr/rollout_is_ratio_fraction_high:5.643627446261235e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.128725489252247e-05 - rollout_corr/rollout_is_max:3.966233491897583 - rollout_corr/rollout_is_min:0.06101220101118088 - rollout_corr/rollout_is_std:0.03435536473989487 - rollout_corr/rollout_is_eff_sample_size:0.9988211002691073 - rollout_corr/rollout_is_seq_mean:0.9998989105224609 - rollout_corr/rollout_is_seq_std:0.0017780093476176262 - rollout_corr/rollout_is_seq_max:1.012873888015747 - rollout_corr/rollout_is_seq_min:0.9941136837005615 - rollout_corr/rollout_is_seq_max_deviation:0.01287388801574707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0185752864927053) - rollout_corr/training_log_ppl:np.float64(0.6845818213769235) - rollout_corr/kl:np.float64(0.0008537062007487606) - rollout_corr/k3_kl:np.float64(0.0007372877449256521) - rollout_corr/rollout_ppl:np.float64(2.0169089869596064) - rollout_corr/rollout_log_ppl:np.float64(0.6837281162734143) - rollout_corr/log_ppl_diff:np.float64(0.0008537051035091281) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014192980015650392) - rollout_corr/log_ppl_diff_max:np.float64(0.015641063451766968) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030567049980163574) - rollout_corr/ppl_ratio:np.float64(1.0008557450491935) - rollout_corr/chi2_token:np.float64(0.0012827848549932241) - rollout_corr/chi2_seq:np.float64(1.2393478201702237) - actor/pg_loss:np.float64(-2.0556850358843803e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026034799202534487) - actor/ppo_kl:np.float64(7.98297634219125e-05) - actor/pg_clipfrac_lower:np.float64(1.187310044770129e-05) - actor/grad_norm:np.float64(0.25147897377610207) - perf/mfu/actor:np.float64(0.11886404252645519) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.61107635498047) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010137365898117423 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010137365898117423 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:692.15234375 - response_length/max:1935.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:692.15234375 - response_length_non_aborted/max:1935.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.903677523136139e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8212964609265327) - timing_s/agent_loop/generate_sequences/max:np.float64(16.11359540745616) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.880805795379274) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.11359540745616) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:371 - timing_s/agent_loop/slowest/response_length:1935 - timing_s/gen:22.885224971920252 - timing_s/reward:0.0695581566542387 - timing_s/old_log_prob:3.2812732346355915 - timing_s/adv:0.15897942706942558 - timing_s/update_actor:13.396265247836709 - timing_s/step:39.86858329549432 - timing_s/stop_profile:0.00011085905134677887 - timing_per_token_ms/adv:0.0006462394446882633 - timing_per_token_ms/gen:0.1291556849496885 - timing_per_token_ms/update_actor:0.05445481326887734 - perf/total_num_tokens:246007 - perf/time_per_step:39.86858329549432 - perf/throughput:771.3059371105182 (TaskRunner pid=2224264) Training Progress: 88%|████████▊ | 88/100 [1:26:54<08:44, 43.68s/it] (TaskRunner pid=2224264) step:89 - global_seqlen/min:30256 - global_seqlen/max:40801 - global_seqlen/minmax_diff:10545 - global_seqlen/balanced_min:34912 - global_seqlen/balanced_max:34937 - global_seqlen/mean:34929.25 - actor/entropy:0.7014224529266357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5826672315597534 - training/rollout_probs_diff_mean:0.004312381613999605 - training/rollout_probs_diff_std:0.008647686801850796 - training/rollout_actor_probs_pearson_corr:0.9996581673622131 - rollout_corr/rollout_is_mean:1.0000931024551392 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.952648199221585e-05 - rollout_corr/rollout_is_max:1.9627834558486938 - rollout_corr/rollout_is_min:0.3244985342025757 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988213381259312 - rollout_corr/rollout_is_seq_mean:1.000161051750183 - rollout_corr/rollout_is_seq_std:0.0014398518251255155 - rollout_corr/rollout_is_seq_max:1.0050846338272095 - rollout_corr/rollout_is_seq_min:0.9958447813987732 - rollout_corr/rollout_is_seq_max_deviation:0.005084633827209473 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.095038112718612) - rollout_corr/training_log_ppl:np.float64(0.7223394099273719) - rollout_corr/kl:np.float64(0.0004870060852217861) - rollout_corr/k3_kl:np.float64(0.0006450895384659816) - rollout_corr/rollout_ppl:np.float64(2.094029687810689) - rollout_corr/rollout_log_ppl:np.float64(0.7218523986521177) - rollout_corr/log_ppl_diff:np.float64(0.00048701127525418997) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011224675690755248) - rollout_corr/log_ppl_diff_max:np.float64(0.004818141460418701) - rollout_corr/log_ppl_diff_min:np.float64(-0.003881394863128662) - rollout_corr/ppl_ratio:np.float64(1.0004880586639047) - rollout_corr/chi2_token:np.float64(0.0016097698826342821) - rollout_corr/chi2_seq:np.float64(1.3918255360331386) - actor/pg_loss:np.float64(7.724866736680269e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001636164154206199) - actor/ppo_kl:np.float64(3.664902180844365e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23648764193058014) - perf/mfu/actor:np.float64(0.1338068099945285) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.57841873168945) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012478643329814076 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0012478643329814076 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:800.1953125 - response_length/max:1701.0 - response_length/min:290.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:800.1953125 - response_length_non_aborted/max:1701.0 - response_length_non_aborted/min:290.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.920712232589722e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.7672608997672796) - timing_s/agent_loop/generate_sequences/max:np.float64(15.327744403854012) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.27638334871881) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.327744403854012) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:223 - timing_s/agent_loop/slowest/response_length:1701 - timing_s/gen:21.85275011509657 - timing_s/reward:0.08627365715801716 - timing_s/old_log_prob:3.273784726858139 - timing_s/adv:0.1331963650882244 - timing_s/update_actor:13.806754915043712 - timing_s/step:39.24401411600411 - timing_s/stop_profile:0.00011419318616390228 - timing_per_token_ms/adv:0.0004766648478289128 - timing_per_token_ms/gen:0.10667683727164544 - timing_per_token_ms/update_actor:0.04940971719634587 - perf/total_num_tokens:279434 - perf/time_per_step:39.24401411600411 - perf/throughput:890.0529363981524 (TaskRunner pid=2224264) Training Progress: 89%|████████▉ | 89/100 [1:27:34<07:45, 42.36s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product molecule: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a nitrogen-containing heterocyclic compound with the following structural features: (TaskRunner pid=2224264) - A ring with the formula Nc1ccn(...)n1, suggesting a pyrrole-like ring (a five-membered ring with a nitrogen). (TaskRunner pid=2224264) - A substituent on the nitrogen: **Cc2ccc(Cl)cc2C(F)(F)F**, which is a benzene ring with a chlorine and a trifluoromethyl group attached to the same carbon. (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are to determine which of the four options is the **only correct reactant** used in the synthesis of this product. Let's analyze the options one by one. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule has a **benzene ring** (c2ccccc2), a **carbonyl group** (O=C), and a **trifluoromethyl group** (C(F)(F)F) attached to a carbon that is also connected to a **chlorine-substituted benzene ring**. Additionally, it contains a **pyrrole-like ring** with a nitrogen (N1) and a substituent **Cc2ccc(Cl)cc2C(F)(F)F**, which is **identical to the substituent in the product**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This matches the **product** almost exactly, and it appears to be a **precursor** or **reactant** that could form the product via a reaction such as a **condensation or cyclization**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule has a **benzene ring** and a **trifluoromethyl group is missing**, and the nitrogen is attached to a **chlorine** (C(Cl)N1), not a **trifluoromethyl** group. It also has a **different structure** (e.g., a different substituent on the nitrogen), and the ring structure doesn't match the product's pyrrole-like ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Not a match.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **benzene ring**, a **trifluoromethyl group** (C(F)(F)F), and a **chlorine** (on a benzene ring). However, the **substituent on the nitrogen is different** (it has an **ether oxygen** and a different aromatic ring), and the **structure of the ring is different** from the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Not a match.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **benzene ring**, but the **substituent on the nitrogen is not the same** as in the product (it has an ethyl group, not the same benzene ring with a chlorine). The **ring structure is also different**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Not a match.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Analysis: (TaskRunner pid=2224264) Only **Option A** has: (TaskRunner pid=2224264) - A **pyrrole-like ring** with a nitrogen. (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F). (TaskRunner pid=2224264) - A **chlorine-substituted benzene ring** (Cc2ccc(Cl)cc2C(F)(F)F). (TaskRunner pid=2224264) - The **same substituent as in the product**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Therefore, **Option A** is the only molecule that is a **valid reactant** for the synthesis of the given product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2224264) step:90 - global_seqlen/min:26503 - global_seqlen/max:35349 - global_seqlen/minmax_diff:8846 - global_seqlen/balanced_min:32189 - global_seqlen/balanced_max:32202 - global_seqlen/mean:32197.5 - actor/entropy:0.6767091751098633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24459987878799438 - training/rollout_probs_diff_mean:0.004204816184937954 - training/rollout_probs_diff_std:0.008432798087596893 - training/rollout_actor_probs_pearson_corr:0.999668538570404 - rollout_corr/rollout_is_mean:1.000195026397705 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.363540822145296e-06 - rollout_corr/rollout_is_max:1.6443707942962646 - rollout_corr/rollout_is_min:0.3406277298927307 - rollout_corr/rollout_is_std:0.033881932497024536 - rollout_corr/rollout_is_eff_sample_size:0.9988539255817266 - rollout_corr/rollout_is_seq_mean:1.0002124309539795 - rollout_corr/rollout_is_seq_std:0.0013589308364316821 - rollout_corr/rollout_is_seq_max:1.0058317184448242 - rollout_corr/rollout_is_seq_min:0.9959684610366821 - rollout_corr/rollout_is_seq_max_deviation:0.005831718444824219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.031145083718002) - rollout_corr/training_log_ppl:np.float64(0.6887705723638646) - rollout_corr/kl:np.float64(0.0006024212293984088) - rollout_corr/k3_kl:np.float64(0.0006607583559343766) - rollout_corr/rollout_ppl:np.float64(2.0299026197753847) - rollout_corr/rollout_log_ppl:np.float64(0.6881681511295028) - rollout_corr/log_ppl_diff:np.float64(0.0006024212343618274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012092863908037543) - rollout_corr/log_ppl_diff_max:np.float64(0.008667349815368652) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036773681640625) - rollout_corr/ppl_ratio:np.float64(1.000603698194027) - rollout_corr/chi2_token:np.float64(0.0014522264245897532) - rollout_corr/chi2_seq:np.float64(0.6241840105503798) - actor/pg_loss:np.float64(-5.9257843531668186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015337120976255392) - actor/ppo_kl:np.float64(0.00021126134707571964) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18526915088295937) - perf/mfu/actor:np.float64(0.11914403018889852) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.8291244506836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6815476190476191) - val-aux/sciknoweval/reward/std@16:np.float64(0.2156221846747797) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7688476190476189) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18144705579138373) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5926619047619047) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1824273460834292) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6801857142857143) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2154753727607372) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8427380952380952) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13509620467575745) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5238857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13671434851677694) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7026523809523809) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1762779201364988) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.895695238095238) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08638656437772123) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47133333333333327) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09762932212980469) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7126714285714286) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13634398316845442) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9269000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.048177977724428236) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42947619047619046) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06802419225042791) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7179285714285714) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10690355682237213) - val-aux/sciknoweval/score/mean@16:np.float64(0.6815476190476191) - val-aux/sciknoweval/score/std@16:np.float64(0.2156221846747797) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7688476190476189) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18144705579138373) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5926619047619047) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1824273460834292) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6801857142857143) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2154753727607372) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8427380952380952) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13509620467575745) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5238857142857143) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13671434851677694) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7026523809523809) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1762779201364988) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.895695238095238) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08638656437772123) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47133333333333327) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09762932212980469) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7126714285714286) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13634398316845442) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9269000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.048177977724428236) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42947619047619046) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06802419225042791) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7179285714285714) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10690355682237213) - val-core/sciknoweval/acc/mean@16:np.float64(0.6815476190476191) - val-aux/sciknoweval/acc/std@16:np.float64(0.2156221846747797) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7688476190476189) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18144705579138373) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5926619047619047) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1824273460834292) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6801857142857143) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2154753727607372) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8427380952380952) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13509620467575745) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5238857142857143) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13671434851677694) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7026523809523809) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1762779201364988) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.895695238095238) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08638656437772123) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47133333333333327) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09762932212980469) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7126714285714286) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13634398316845442) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9269000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.048177977724428236) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42947619047619046) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06802419225042791) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7179285714285714) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10690355682237213) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9767857142857143) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0671951594177618) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9968238095238093) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.022641622754649007) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9569095238095239) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08670860027074634) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9766666666666668) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06717160756445564) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998523809523809) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0023879182972022734) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9241857142857143) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.10421760940450413) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9911857142857143) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.04162781171103064) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8778714285714286) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11040129964532797) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9977285714285715) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.020137431304512083) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8244380952380953) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.09488164505205152) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997952380952381) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00450120576193258) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026113740168511868 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026113740168511868 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:728.296875 - response_length/max:1524.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:728.296875 - response_length_non_aborted/max:1524.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011400692164897919 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4655274618417025) - timing_s/agent_loop/generate_sequences/max:np.float64(14.063222439959645) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.401196123711998) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.063222439959645) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:371 - timing_s/agent_loop/slowest/response_length:1524 - timing_s/gen:20.9882550816983 - timing_s/reward:0.07745247334241867 - timing_s/old_log_prob:3.1980783604085445 - timing_s/adv:0.11822893470525742 - timing_s/update_actor:14.292423715814948 - timing_s/step:38.767167622223496 - timing_s/testing:206.41960541717708 - timing_s/save_checkpoint:12.91796376183629 - timing_s/stop_profile:0.00014068372547626495 - timing_per_token_ms/adv:0.00045899889240335977 - timing_per_token_ms/gen:0.11257136234847084 - timing_per_token_ms/update_actor:0.05548731934084536 - perf/total_num_tokens:257580 - perf/time_per_step:38.767167622223496 - perf/throughput:830.5352692710675 (TaskRunner pid=2224264) Training Progress: 90%|█████████ | 90/100 [1:31:52<17:51, 107.11s/it] (TaskRunner pid=2224264) step:91 - global_seqlen/min:27956 - global_seqlen/max:38739 - global_seqlen/minmax_diff:10783 - global_seqlen/balanced_min:31665 - global_seqlen/balanced_max:31695 - global_seqlen/mean:31683.5 - actor/entropy:0.7154297232627869 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9744127988815308 - training/rollout_probs_diff_mean:0.004367751069366932 - training/rollout_probs_diff_std:0.00891902670264244 - training/rollout_actor_probs_pearson_corr:0.9995865821838379 - rollout_corr/rollout_is_mean:0.9999821186065674 - rollout_corr/rollout_is_ratio_fraction_high:5.548773515329231e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.1097547030658461e-05 - rollout_corr/rollout_is_max:2.104722261428833 - rollout_corr/rollout_is_min:0.00197325786575675 - rollout_corr/rollout_is_std:0.03477783128619194 - rollout_corr/rollout_is_eff_sample_size:0.9987919636650865 - rollout_corr/rollout_is_seq_mean:0.9999819993972778 - rollout_corr/rollout_is_seq_std:0.0012956903083249927 - rollout_corr/rollout_is_seq_max:1.00333833694458 - rollout_corr/rollout_is_seq_min:0.9953454732894897 - rollout_corr/rollout_is_seq_max_deviation:0.004654526710510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.104408088605851) - rollout_corr/training_log_ppl:np.float64(0.7243244127603248) - rollout_corr/kl:np.float64(0.0006612393046498966) - rollout_corr/k3_kl:np.float64(0.0006583722558843874) - rollout_corr/rollout_ppl:np.float64(2.102920087054372) - rollout_corr/rollout_log_ppl:np.float64(0.7236631693085656) - rollout_corr/log_ppl_diff:np.float64(0.0006612434517592192) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012168984394520521) - rollout_corr/log_ppl_diff_max:np.float64(0.00693821907043457) - rollout_corr/log_ppl_diff_min:np.float64(-0.0027844905853271484) - rollout_corr/ppl_ratio:np.float64(1.0006624751258641) - rollout_corr/chi2_token:np.float64(0.001317617017775774) - rollout_corr/chi2_seq:np.float64(1.2634925008751452) - actor/pg_loss:np.float64(0.0001394905848428607) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015355198502220446) - actor/ppo_kl:np.float64(-4.81430525312021e-06) - actor/pg_clipfrac_lower:np.float64(4.3841191654792055e-06) - actor/grad_norm:np.float64(0.22441749274730682) - perf/mfu/actor:np.float64(0.1235221491000339) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(107.04840850830078) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006852735532447696 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006852735532447696 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:703.984375 - response_length/max:1848.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:703.984375 - response_length_non_aborted/max:1848.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.789582014083862e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.103301614522934) - timing_s/agent_loop/generate_sequences/max:np.float64(15.148089375346899) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.013559678431193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.148089375346899) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:1848 - timing_s/gen:21.60629509203136 - timing_s/reward:0.07231813855469227 - timing_s/old_log_prob:3.4730070438236 - timing_s/adv:0.13708950020372868 - timing_s/update_actor:13.167173700407147 - timing_s/step:38.57017913646996 - timing_s/stop_profile:0.00017870403826236725 - timing_per_token_ms/adv:0.0005408552566940548 - timing_per_token_ms/gen:0.11988844241500034 - timing_per_token_ms/update_actor:0.051948071158517635 - perf/total_num_tokens:253468 - perf/time_per_step:38.57017913646996 - perf/throughput:821.4506831274145 (TaskRunner pid=2224264) Training Progress: 91%|█████████ | 91/100 [1:32:30<12:59, 86.56s/it] (TaskRunner pid=2224264) step:92 - global_seqlen/min:28907 - global_seqlen/max:37134 - global_seqlen/minmax_diff:8227 - global_seqlen/balanced_min:32432 - global_seqlen/balanced_max:32477 - global_seqlen/mean:32465.625 - actor/entropy:0.7178817987442017 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2975451946258545 - training/rollout_probs_diff_mean:0.004235805943608284 - training/rollout_probs_diff_std:0.008502726443111897 - training/rollout_actor_probs_pearson_corr:0.9996208548545837 - rollout_corr/rollout_is_mean:0.9999630451202393 - rollout_corr/rollout_is_ratio_fraction_high:5.372733539843466e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1497116088867188 - rollout_corr/rollout_is_min:0.550010621547699 - rollout_corr/rollout_is_std:0.03401536867022514 - rollout_corr/rollout_is_eff_sample_size:0.9988440539746302 - rollout_corr/rollout_is_seq_mean:0.9999423623085022 - rollout_corr/rollout_is_seq_std:0.0013055680319666862 - rollout_corr/rollout_is_seq_max:1.004227876663208 - rollout_corr/rollout_is_seq_min:0.9956101775169373 - rollout_corr/rollout_is_seq_max_deviation:0.004389822483062744 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.064156681764871) - rollout_corr/training_log_ppl:np.float64(0.7071324991993606) - rollout_corr/kl:np.float64(0.0008166503134958703) - rollout_corr/k3_kl:np.float64(0.0006099401913388647) - rollout_corr/rollout_ppl:np.float64(2.0623925644904375) - rollout_corr/rollout_log_ppl:np.float64(0.7063158513628878) - rollout_corr/log_ppl_diff:np.float64(0.0008166478364728391) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012665537069551647) - rollout_corr/log_ppl_diff_max:np.float64(0.007030129432678223) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036208629608154297) - rollout_corr/ppl_ratio:np.float64(1.0008179140277207) - rollout_corr/chi2_token:np.float64(0.0008041651453822851) - rollout_corr/chi2_seq:np.float64(0.6329832167830318) - actor/pg_loss:np.float64(6.987596862018108e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012492035489231057) - actor/ppo_kl:np.float64(0.0001699282564437965) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19231200218200684) - perf/mfu/actor:np.float64(0.12487351766630518) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.44847011566162) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004429147113114595 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004429147113114595 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:727.05078125 - response_length/max:1984.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:727.05078125 - response_length_non_aborted/max:1984.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019785761833190918 - timing_s/agent_loop/generate_sequences/min:np.float64(2.329558404162526) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53738614730537) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.280053526119445) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53738614730537) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1984 - timing_s/gen:23.22112664207816 - timing_s/reward:0.07667875476181507 - timing_s/old_log_prob:3.3302839752286673 - timing_s/adv:0.12147887982428074 - timing_s/update_actor:13.518468294292688 - timing_s/step:40.354144191369414 - timing_s/stop_profile:0.00012041814625263214 - timing_per_token_ms/adv:0.00046772116594197993 - timing_per_token_ms/gen:0.12476092218712242 - timing_per_token_ms/update_actor:0.0520491608212251 - perf/total_num_tokens:259725 - perf/time_per_step:40.354144191369414 - perf/throughput:804.5177428627878 (TaskRunner pid=2224264) Training Progress: 92%|█████████▏| 92/100 [1:33:11<09:41, 72.72s/it] (TaskRunner pid=2224264) step:93 - global_seqlen/min:23013 - global_seqlen/max:34946 - global_seqlen/minmax_diff:11933 - global_seqlen/balanced_min:27908 - global_seqlen/balanced_max:27913 - global_seqlen/mean:27910.875 - actor/entropy:0.7500928044319153 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1930849552154541 - training/rollout_probs_diff_mean:0.004663003142923117 - training/rollout_probs_diff_std:0.008796103298664093 - training/rollout_actor_probs_pearson_corr:0.9996201992034912 - rollout_corr/rollout_is_mean:0.999956488609314 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6296987533569336 - rollout_corr/rollout_is_min:0.6042289137840271 - rollout_corr/rollout_is_std:0.035632822662591934 - rollout_corr/rollout_is_eff_sample_size:0.9987316741318075 - rollout_corr/rollout_is_seq_mean:0.9998897910118103 - rollout_corr/rollout_is_seq_std:0.0015105620259419084 - rollout_corr/rollout_is_seq_max:1.0058465003967285 - rollout_corr/rollout_is_seq_min:0.9952189922332764 - rollout_corr/rollout_is_seq_max_deviation:0.005846500396728516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1263672169297934) - rollout_corr/training_log_ppl:np.float64(0.7347684952546842) - rollout_corr/kl:np.float64(0.0007506872085336624) - rollout_corr/k3_kl:np.float64(0.0006735208150985272) - rollout_corr/rollout_ppl:np.float64(2.124785815831274) - rollout_corr/rollout_log_ppl:np.float64(0.7340178099111654) - rollout_corr/log_ppl_diff:np.float64(0.0007506853435188532) - rollout_corr/log_ppl_abs_diff:np.float64(0.001359031768515706) - rollout_corr/log_ppl_diff_max:np.float64(0.005597174167633057) - rollout_corr/log_ppl_diff_min:np.float64(-0.003313422203063965) - rollout_corr/ppl_ratio:np.float64(1.000752320047468) - rollout_corr/chi2_token:np.float64(0.0011984859593212605) - rollout_corr/chi2_seq:np.float64(0.6925807157531381) - actor/pg_loss:np.float64(-4.779419396072626e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.594229848313262e-05) - actor/ppo_kl:np.float64(-7.275951552143667e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18249434977769852) - perf/mfu/actor:np.float64(0.11041731672624022) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.24386596679688) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007838345132768154 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007838345132768154 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:605.74609375 - response_length/max:1369.0 - response_length/min:190.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:605.74609375 - response_length_non_aborted/max:1369.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018016062676906586 - timing_s/agent_loop/generate_sequences/min:np.float64(2.355575904250145) - timing_s/agent_loop/generate_sequences/max:np.float64(11.529610631987453) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.815270844308543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.529610631987453) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:1369 - timing_s/gen:18.318352431058884 - timing_s/reward:0.07145974412560463 - timing_s/old_log_prob:3.1092032585293055 - timing_s/adv:0.12323503755033016 - timing_s/update_actor:13.111440777778625 - timing_s/step:34.81759689189494 - timing_s/stop_profile:2.6732683181762695e-05 - timing_per_token_ms/adv:0.0005519131769889432 - timing_per_token_ms/gen:0.11812880829464492 - timing_per_token_ms/update_actor:0.05872012601619721 - perf/total_num_tokens:223287 - perf/time_per_step:34.81759689189494 - perf/throughput:801.6312868076565 (TaskRunner pid=2224264) Training Progress: 93%|█████████▎| 93/100 [1:33:46<07:09, 61.36s/it] (TaskRunner pid=2224264) step:94 - global_seqlen/min:24960 - global_seqlen/max:37338 - global_seqlen/minmax_diff:12378 - global_seqlen/balanced_min:30359 - global_seqlen/balanced_max:30372 - global_seqlen/mean:30366.25 - actor/entropy:0.7063392400741577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3128166198730469 - training/rollout_probs_diff_mean:0.004224883858114481 - training/rollout_probs_diff_std:0.008449923247098923 - training/rollout_actor_probs_pearson_corr:0.9996349811553955 - rollout_corr/rollout_is_mean:0.9998794198036194 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6113572100002784e-06 - rollout_corr/rollout_is_max:1.5920956134796143 - rollout_corr/rollout_is_min:0.19120460748672485 - rollout_corr/rollout_is_std:0.03371969982981682 - rollout_corr/rollout_is_eff_sample_size:0.99886415419973 - rollout_corr/rollout_is_seq_mean:0.9998659491539001 - rollout_corr/rollout_is_seq_std:0.0013268847251310945 - rollout_corr/rollout_is_seq_max:1.0026715993881226 - rollout_corr/rollout_is_seq_min:0.9956828355789185 - rollout_corr/rollout_is_seq_max_deviation:0.004317164421081543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.097409463953227) - rollout_corr/training_log_ppl:np.float64(0.7194324630545452) - rollout_corr/kl:np.float64(0.0006619715687250505) - rollout_corr/k3_kl:np.float64(0.0006161111765550231) - rollout_corr/rollout_ppl:np.float64(2.0959334070794284) - rollout_corr/rollout_log_ppl:np.float64(0.7187704919488169) - rollout_corr/log_ppl_diff:np.float64(0.0006619711057282984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012567761004902422) - rollout_corr/log_ppl_diff_max:np.float64(0.004305779933929443) - rollout_corr/log_ppl_diff_min:np.float64(-0.004627227783203125) - rollout_corr/ppl_ratio:np.float64(1.0006632735021412) - rollout_corr/chi2_token:np.float64(0.00113762472756207) - rollout_corr/chi2_seq:np.float64(2.2418907159008086) - actor/pg_loss:np.float64(9.933649562299252e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.07081788580399e-05) - actor/ppo_kl:np.float64(-6.808040846806307e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1753209587186575) - perf/mfu/actor:np.float64(0.11873190434052154) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.19572830200195) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006172493100166321 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006172493100166321 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:696.1328125 - response_length/max:1462.0 - response_length/min:227.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:696.1328125 - response_length_non_aborted/max:1462.0 - response_length_non_aborted/min:227.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.6177377700805664e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.743150135502219) - timing_s/agent_loop/generate_sequences/max:np.float64(13.47325548902154) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.079402086157643) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.47325548902154) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:1462 - timing_s/gen:20.203036999329925 - timing_s/reward:0.07774028927087784 - timing_s/old_log_prob:3.2072019651532173 - timing_s/adv:0.12383914738893509 - timing_s/update_actor:13.169512040913105 - timing_s/step:36.86546222306788 - timing_s/stop_profile:0.00011588446795940399 - timing_per_token_ms/adv:0.0005097729691225254 - timing_per_token_ms/gen:0.11336646091313576 - timing_per_token_ms/update_actor:0.054211139179653006 - perf/total_num_tokens:242930 - perf/time_per_step:36.86546222306788 - perf/throughput:823.7045779124636 (TaskRunner pid=2224264) Training Progress: 94%|█████████▍| 94/100 [1:34:23<05:24, 54.02s/it] (TaskRunner pid=2224264) step:95 - global_seqlen/min:20976 - global_seqlen/max:34422 - global_seqlen/minmax_diff:13446 - global_seqlen/balanced_min:29028 - global_seqlen/balanced_max:29035 - global_seqlen/mean:29031.75 - actor/entropy:0.712871789932251 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2599616050720215 - training/rollout_probs_diff_mean:0.004302844405174255 - training/rollout_probs_diff_std:0.008476506918668747 - training/rollout_actor_probs_pearson_corr:0.9996457099914551 - rollout_corr/rollout_is_mean:1.0001041889190674 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.065090474294266e-06 - rollout_corr/rollout_is_max:1.6687859296798706 - rollout_corr/rollout_is_min:0.4844895303249359 - rollout_corr/rollout_is_std:0.03429459035396576 - rollout_corr/rollout_is_eff_sample_size:0.9988256195681333 - rollout_corr/rollout_is_seq_mean:1.0001366138458252 - rollout_corr/rollout_is_seq_std:0.0015432714717462659 - rollout_corr/rollout_is_seq_max:1.005467414855957 - rollout_corr/rollout_is_seq_min:0.9945797324180603 - rollout_corr/rollout_is_seq_max_deviation:0.005467414855957031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0434681265614927) - rollout_corr/training_log_ppl:np.float64(0.6912423392059281) - rollout_corr/kl:np.float64(0.0005482688078317643) - rollout_corr/k3_kl:np.float64(0.0006431571834468741) - rollout_corr/rollout_ppl:np.float64(2.0423742262646556) - rollout_corr/rollout_log_ppl:np.float64(0.6906940678600222) - rollout_corr/log_ppl_diff:np.float64(0.0005482713459059596) - rollout_corr/log_ppl_abs_diff:np.float64(0.001258363830856979) - rollout_corr/log_ppl_diff_max:np.float64(0.005878806114196777) - rollout_corr/log_ppl_diff_min:np.float64(-0.005048781633377075) - rollout_corr/ppl_ratio:np.float64(1.000549679622054) - rollout_corr/chi2_token:np.float64(0.0014886993449181318) - rollout_corr/chi2_seq:np.float64(3.0285099220927805) - actor/pg_loss:np.float64(-0.000160741969011724) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010068776555272052) - actor/ppo_kl:np.float64(7.83663859635908e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23990549892187119) - perf/mfu/actor:np.float64(0.11424140158128934) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.15832138061523) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014393975958228111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014393975958228111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.0546875 - response_length/max:1596.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.0546875 - response_length_non_aborted/max:1596.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.495019912719727e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.937621222808957) - timing_s/agent_loop/generate_sequences/max:np.float64(13.714711025357246) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.356954876297095) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.714711025357246) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:1596 - timing_s/gen:20.76876393146813 - timing_s/reward:0.07113218307495117 - timing_s/old_log_prob:3.2435854095965624 - timing_s/adv:0.1196875274181366 - timing_s/update_actor:13.214123142883182 - timing_s/step:37.502956656739116 - timing_s/stop_profile:0.00011238828301429749 - timing_per_token_ms/adv:0.0005153303168864115 - timing_per_token_ms/gen:0.12596443389335224 - timing_per_token_ms/update_actor:0.056895136974532974 - perf/total_num_tokens:232254 - perf/time_per_step:37.502956656739116 - perf/throughput:774.1189652251891 (TaskRunner pid=2224264) Training Progress: 95%|█████████▌| 95/100 [1:35:00<04:05, 49.08s/it] (TaskRunner pid=2224264) step:96 - global_seqlen/min:27198 - global_seqlen/max:35708 - global_seqlen/minmax_diff:8510 - global_seqlen/balanced_min:32177 - global_seqlen/balanced_max:32213 - global_seqlen/mean:32201.875 - actor/entropy:0.7203529477119446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6764640808105469 - training/rollout_probs_diff_mean:0.004217594396322966 - training/rollout_probs_diff_std:0.008475503884255886 - training/rollout_actor_probs_pearson_corr:0.999649703502655 - rollout_corr/rollout_is_mean:1.0000306367874146 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.28119744558353e-06 - rollout_corr/rollout_is_max:1.7523001432418823 - rollout_corr/rollout_is_min:0.0011872303439304233 - rollout_corr/rollout_is_std:0.03390303626656532 - rollout_corr/rollout_is_eff_sample_size:0.9988520226061409 - rollout_corr/rollout_is_seq_mean:0.9999527931213379 - rollout_corr/rollout_is_seq_std:0.0014494654024019837 - rollout_corr/rollout_is_seq_max:1.0033957958221436 - rollout_corr/rollout_is_seq_min:0.9943961501121521 - rollout_corr/rollout_is_seq_max_deviation:0.0056038498878479 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0765154217369854) - rollout_corr/training_log_ppl:np.float64(0.7100228805211373) - rollout_corr/kl:np.float64(0.0007996692028076069) - rollout_corr/k3_kl:np.float64(0.0006997747711352531) - rollout_corr/rollout_ppl:np.float64(2.074850105214864) - rollout_corr/rollout_log_ppl:np.float64(0.7092232063878328) - rollout_corr/log_ppl_diff:np.float64(0.0007996741333045065) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013392329565249383) - rollout_corr/log_ppl_diff_max:np.float64(0.006226241588592529) - rollout_corr/log_ppl_diff_min:np.float64(-0.002697765827178955) - rollout_corr/ppl_ratio:np.float64(1.0008012598846108) - rollout_corr/chi2_token:np.float64(0.0011850390583276749) - rollout_corr/chi2_seq:np.float64(0.8988333453889936) - actor/pg_loss:np.float64(-6.152375135570765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024159732652151433) - actor/ppo_kl:np.float64(0.00012709150854451137) - actor/pg_clipfrac_lower:np.float64(4.822531082027126e-06) - actor/grad_norm:np.float64(0.21260051056742668) - perf/mfu/actor:np.float64(0.12350445771061563) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.11673736572266) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01730252243578434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01730252243578434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:739.65234375 - response_length/max:1942.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:739.65234375 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010816939175128937 - timing_s/agent_loop/generate_sequences/min:np.float64(1.915278973057866) - timing_s/agent_loop/generate_sequences/max:np.float64(16.20021486841142) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.129882725828793) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.20021486841142) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1903 - timing_s/gen:23.026053931564093 - timing_s/reward:0.07937229797244072 - timing_s/old_log_prob:3.2644342444837093 - timing_s/adv:0.1240159124135971 - timing_s/update_actor:13.31886836886406 - timing_s/step:39.908353347331285 - timing_s/stop_profile:0.00011824257671833038 - timing_per_token_ms/adv:0.00048140019957532403 - timing_per_token_ms/gen:0.12160513507488259 - timing_per_token_ms/update_actor:0.05170067103570856 - perf/total_num_tokens:257615 - perf/time_per_step:39.908353347331285 - perf/throughput:806.8956070359484 (TaskRunner pid=2224264) Training Progress: 96%|█████████▌| 96/100 [1:35:40<03:05, 46.34s/it] (TaskRunner pid=2224264) step:97 - global_seqlen/min:31313 - global_seqlen/max:43694 - global_seqlen/minmax_diff:12381 - global_seqlen/balanced_min:35934 - global_seqlen/balanced_max:35960 - global_seqlen/mean:35950.125 - actor/entropy:0.7323668003082275 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915082573890686 - training/rollout_probs_diff_mean:0.004103551618754864 - training/rollout_probs_diff_std:0.008156946860253811 - training/rollout_actor_probs_pearson_corr:0.9996232986450195 - rollout_corr/rollout_is_mean:0.9999871850013733 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.7118023758230265e-06 - rollout_corr/rollout_is_max:1.6210092306137085 - rollout_corr/rollout_is_min:0.4609464108943939 - rollout_corr/rollout_is_std:0.03369848057627678 - rollout_corr/rollout_is_eff_sample_size:0.9988657004043994 - rollout_corr/rollout_is_seq_mean:1.000003695487976 - rollout_corr/rollout_is_seq_std:0.0014436065685003996 - rollout_corr/rollout_is_seq_max:1.0053772926330566 - rollout_corr/rollout_is_seq_min:0.9946680665016174 - rollout_corr/rollout_is_seq_max_deviation:0.005377292633056641 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1260625598952174) - rollout_corr/training_log_ppl:np.float64(0.7336573095526546) - rollout_corr/kl:np.float64(0.0006526818591758854) - rollout_corr/k3_kl:np.float64(0.0006329746193500796) - rollout_corr/rollout_ppl:np.float64(2.124627975281328) - rollout_corr/rollout_log_ppl:np.float64(0.7330046314746141) - rollout_corr/log_ppl_diff:np.float64(0.0006526780780404806) - rollout_corr/log_ppl_abs_diff:np.float64(0.001235815929248929) - rollout_corr/log_ppl_diff_max:np.float64(0.00612562894821167) - rollout_corr/log_ppl_diff_min:np.float64(-0.00426250696182251) - rollout_corr/ppl_ratio:np.float64(1.0006539451424032) - rollout_corr/chi2_token:np.float64(0.0012377453967928886) - rollout_corr/chi2_seq:np.float64(0.6915090307593346) - actor/pg_loss:np.float64(-3.366696182638407e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016491950168529002) - actor/ppo_kl:np.float64(6.479799488445792e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20394239574670792) - perf/mfu/actor:np.float64(0.135031184027903) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.04956817626953) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012476264499127865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012476264499127865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:829.03515625 - response_length/max:2245.0 - response_length/min:215.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:829.03515625 - response_length_non_aborted/max:2245.0 - response_length_non_aborted/min:215.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010967254638671875 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4484062399715185) - timing_s/agent_loop/generate_sequences/max:np.float64(18.13408470340073) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.178321859493735) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.13408470340073) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:2245 - timing_s/gen:24.973692329600453 - timing_s/reward:0.08095383085310459 - timing_s/old_log_prob:3.3533995244652033 - timing_s/adv:0.12184758298099041 - timing_s/update_actor:13.665139958262444 - timing_s/step:42.28041913919151 - timing_s/stop_profile:4.5822933316230774e-05 - timing_per_token_ms/adv:0.0004236688432272155 - timing_per_token_ms/gen:0.11767110830832365 - timing_per_token_ms/update_actor:0.047514229638500716 - perf/total_num_tokens:287601 - perf/time_per_step:42.28041913919151 - perf/throughput:850.2783494564818 (TaskRunner pid=2224264) Training Progress: 97%|█████████▋| 97/100 [1:36:22<02:15, 45.13s/it] (TaskRunner pid=2224264) step:98 - global_seqlen/min:25644 - global_seqlen/max:42510 - global_seqlen/minmax_diff:16866 - global_seqlen/balanced_min:34661 - global_seqlen/balanced_max:34705 - global_seqlen/mean:34687.75 - actor/entropy:0.7501354813575745 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19508248567581177 - training/rollout_probs_diff_mean:0.004144626669585705 - training/rollout_probs_diff_std:0.008178996853530407 - training/rollout_actor_probs_pearson_corr:0.9996928572654724 - rollout_corr/rollout_is_mean:1.0000689029693604 - rollout_corr/rollout_is_ratio_fraction_high:4.974382136424538e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2251296043395996 - rollout_corr/rollout_is_min:0.6142621636390686 - rollout_corr/rollout_is_std:0.03403989225625992 - rollout_corr/rollout_is_eff_sample_size:0.9988425078369757 - rollout_corr/rollout_is_seq_mean:1.000027060508728 - rollout_corr/rollout_is_seq_std:0.001423987210728228 - rollout_corr/rollout_is_seq_max:1.0042808055877686 - rollout_corr/rollout_is_seq_min:0.9920931458473206 - rollout_corr/rollout_is_seq_max_deviation:0.007906854152679443 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1749815698713064) - rollout_corr/training_log_ppl:np.float64(0.7591774334432557) - rollout_corr/kl:np.float64(0.0006937429601876488) - rollout_corr/k3_kl:np.float64(0.0006346612378251848) - rollout_corr/rollout_ppl:np.float64(2.17338493373245) - rollout_corr/rollout_log_ppl:np.float64(0.7584836900932714) - rollout_corr/log_ppl_diff:np.float64(0.0006937433499842882) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011937296949326992) - rollout_corr/log_ppl_diff_max:np.float64(0.004764825105667114) - rollout_corr/log_ppl_diff_min:np.float64(-0.003519713878631592) - rollout_corr/ppl_ratio:np.float64(1.000694976421073) - rollout_corr/chi2_token:np.float64(0.0011504706926643848) - rollout_corr/chi2_seq:np.float64(0.7677441604901105) - actor/pg_loss:np.float64(-0.00023154309019446373) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014778738523091306) - actor/ppo_kl:np.float64(0.00011894125558598034) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22566131502389908) - perf/mfu/actor:np.float64(0.13134675089113737) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.8719253540039) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0074124508537352085 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0074124508537352085 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.2734375 - response_length/max:2121.0 - response_length/min:178.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.2734375 - response_length_non_aborted/max:2121.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.526918590068817e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.268034750595689) - timing_s/agent_loop/generate_sequences/max:np.float64(17.475637735798955) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.994100491654535) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.475637735798955) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:2121 - timing_s/gen:24.36921063065529 - timing_s/reward:0.08015711978077888 - timing_s/old_log_prob:3.372231272980571 - timing_s/adv:0.11694180965423584 - timing_s/update_actor:13.581646379083395 - timing_s/step:41.60994468256831 - timing_s/stop_profile:0.00010863691568374634 - timing_per_token_ms/adv:0.0004214088895007454 - timing_per_token_ms/gen:0.12122176108369541 - timing_per_token_ms/update_actor:0.04894251709567281 - perf/total_num_tokens:277502 - perf/time_per_step:41.60994468256831 - perf/throughput:833.6408583242305 (TaskRunner pid=2224264) Training Progress: 98%|█████████▊| 98/100 [1:37:04<01:28, 44.09s/it] (TaskRunner pid=2224264) step:99 - global_seqlen/min:31172 - global_seqlen/max:42762 - global_seqlen/minmax_diff:11590 - global_seqlen/balanced_min:36579 - global_seqlen/balanced_max:36620 - global_seqlen/mean:36605.125 - actor/entropy:0.7388753890991211 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583175301551819 - training/rollout_probs_diff_mean:0.004119738936424255 - training/rollout_probs_diff_std:0.008252698928117752 - training/rollout_actor_probs_pearson_corr:0.9996559023857117 - rollout_corr/rollout_is_mean:1.0000630617141724 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.395273082191125e-06 - rollout_corr/rollout_is_max:1.7297682762145996 - rollout_corr/rollout_is_min:0.46209102869033813 - rollout_corr/rollout_is_std:0.03351224586367607 - rollout_corr/rollout_is_eff_sample_size:0.998878308098044 - rollout_corr/rollout_is_seq_mean:1.0000331401824951 - rollout_corr/rollout_is_seq_std:0.0012539547169581056 - rollout_corr/rollout_is_seq_max:1.0045143365859985 - rollout_corr/rollout_is_seq_min:0.9951211214065552 - rollout_corr/rollout_is_seq_max_deviation:0.004878878593444824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.149727716576308) - rollout_corr/training_log_ppl:np.float64(0.7441099616116844) - rollout_corr/kl:np.float64(0.0006301436051412423) - rollout_corr/k3_kl:np.float64(0.0006160447873639896) - rollout_corr/rollout_ppl:np.float64(2.1482933391816914) - rollout_corr/rollout_log_ppl:np.float64(0.7434798121103086) - rollout_corr/log_ppl_diff:np.float64(0.000630149501375854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011455799685791135) - rollout_corr/log_ppl_diff_max:np.float64(0.0055887699127197266) - rollout_corr/log_ppl_diff_min:np.float64(-0.00408819317817688) - rollout_corr/ppl_ratio:np.float64(1.0006313105113804) - rollout_corr/chi2_token:np.float64(0.0012098602019250393) - rollout_corr/chi2_seq:np.float64(0.841775347944349) - actor/pg_loss:np.float64(-8.723500650376081e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012781737518707814) - actor/ppo_kl:np.float64(8.237922273224285e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19401156529784203) - perf/mfu/actor:np.float64(0.13594204140089314) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.93074417114258) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006903177127242088 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006903177127242088 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:831.53515625 - response_length/max:1998.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:831.53515625 - response_length_non_aborted/max:1998.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.847611904144287e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.493832629173994) - timing_s/agent_loop/generate_sequences/max:np.float64(17.390929045155644) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.606087130981905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.390929045155644) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:493 - timing_s/agent_loop/slowest/response_length:1964 - timing_s/gen:24.117084549739957 - timing_s/reward:0.08298422396183014 - timing_s/old_log_prob:3.351647572591901 - timing_s/adv:0.12600908987224102 - timing_s/update_actor:14.029597079381347 - timing_s/step:41.793392254039645 - timing_s/stop_profile:0.00011212378740310669 - timing_per_token_ms/adv:0.0004302986599289069 - timing_per_token_ms/gen:0.11329329952478688 - timing_per_token_ms/update_actor:0.04790858206119138 - perf/total_num_tokens:292841 - perf/time_per_step:41.793392254039645 - perf/throughput:875.859149635355 (TaskRunner pid=2224264) Training Progress: 99%|█████████▉| 99/100 [1:37:46<00:43, 43.41s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a target product: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) This is a chemical formula representing a molecule. Let's break it down: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - The main structure is: **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) - This can be interpreted as: a ring with a nitrogen (N) at position 1, and a substituent at position 1 (the "C..." part). (TaskRunner pid=2224264) - The substituent is: **Cc2ccc(Cl)cc2C(F)(F)F** (TaskRunner pid=2224264) This is a benzene ring (c2ccc...), with a chlorine (Cl) on one carbon, and a trifluoromethyl group (C(F)(F)F) on another carbon. The entire group is attached to a carbon (C) which is attached to the nitrogen (N) in the ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now, we are to identify which of the four options is the **only correct reactant** that would be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option A:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This is a complex molecule. (TaskRunner pid=2224264) - It contains a ring (C1) with two double bonds (c2ccccc2), and a carbonyl group (O=C). (TaskRunner pid=2224264) - Then, **C(=O)N1** suggests a carbonyl group attached to a nitrogen (N1). (TaskRunner pid=2224264) - Then, **c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** matches the target product's ring and substituent. (TaskRunner pid=2224264) - So, this option contains the full structure of the target product, possibly as a derivative or a starting point for synthesis. (TaskRunner pid=2224264) ✅ **Possibly a correct reactant.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option B:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a ring with Cl attached. (TaskRunner pid=2224264) - The rest of the structure does not match the substituent **Cc2ccc(Cl)cc2C(F)(F)F**. (TaskRunner pid=2224264) - **Does not match the target structure.** (TaskRunner pid=2224264) ❌ **Not correct.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option C:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This has a ring with a substituent that includes a **C(F)(F)F** group. (TaskRunner pid=2224264) - However, the **Oc2ccc(Cl)c...** is different from **Cc2ccc(Cl)cc2C(F)(F)F**. (TaskRunner pid=2224264) - The structure is not matching the target. (TaskRunner pid=2224264) ❌ **Not correct.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option D:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **C(F)(F)F** group. (TaskRunner pid=2224264) - However, the **CC=C(Cc1ccccc1)** is not matching the **Cc2ccc(Cl)cc2C(F)(F)F** part. (TaskRunner pid=2224264) - No Cl is present in the substituent. (TaskRunner pid=2224264) ❌ **Not correct.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Evaluation: (TaskRunner pid=2224264) Only **Option A** contains the exact substituent **Cc2ccc(Cl)cc2C(F)(F)F**, which is part of the target molecule, and the rest of the structure is consistent with the ring and functional group of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) So, **Option A** is the only correct reactant that could be used in the synthesis of the given product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2224264) step:100 - global_seqlen/min:32181 - global_seqlen/max:40888 - global_seqlen/minmax_diff:8707 - global_seqlen/balanced_min:36112 - global_seqlen/balanced_max:36131 - global_seqlen/mean:36125.5 - actor/entropy:0.7533097267150879 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2046983242034912 - training/rollout_probs_diff_mean:0.0041552940383553505 - training/rollout_probs_diff_std:0.00816031452268362 - training/rollout_actor_probs_pearson_corr:0.9996163845062256 - rollout_corr/rollout_is_mean:0.999994695186615 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.690695732278982e-06 - rollout_corr/rollout_is_max:1.4722437858581543 - rollout_corr/rollout_is_min:0.08076818287372589 - rollout_corr/rollout_is_std:0.03385905548930168 - rollout_corr/rollout_is_eff_sample_size:0.9988548770722385 - rollout_corr/rollout_is_seq_mean:0.9999972581863403 - rollout_corr/rollout_is_seq_std:0.0013772835955023766 - rollout_corr/rollout_is_seq_max:1.0046179294586182 - rollout_corr/rollout_is_seq_min:0.9932000041007996 - rollout_corr/rollout_is_seq_max_deviation:0.0067999958992004395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1957929017953575) - rollout_corr/training_log_ppl:np.float64(0.7637257019523531) - rollout_corr/kl:np.float64(0.0006403183531915602) - rollout_corr/k3_kl:np.float64(0.0006129170444637566) - rollout_corr/rollout_ppl:np.float64(2.194364957511425) - rollout_corr/rollout_log_ppl:np.float64(0.7630853808368556) - rollout_corr/log_ppl_diff:np.float64(0.0006403211154974997) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010970584698952734) - rollout_corr/log_ppl_diff_max:np.float64(0.007486581802368164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030992627143859863) - rollout_corr/ppl_ratio:np.float64(1.000641372287646) - rollout_corr/chi2_token:np.float64(0.0011783752124756575) - rollout_corr/chi2_seq:np.float64(1.0062491311691701) - actor/pg_loss:np.float64(1.9038328900933266e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016279506189675885) - actor/ppo_kl:np.float64(4.644049759150448e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20593710616230965) - perf/mfu/actor:np.float64(0.13613751711674105) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.93215942382812) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6901785714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.21497342883717385) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7771380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17605374457438094) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6025857142857143) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18719434569533389) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6898333333333334) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21523251642703473) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8483380952380953) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.128790863116223) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5304333333333333) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14423738321883778) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7146857142857143) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17575431812173817) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8984857142857143) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08310585797508012) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47553809523809526) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10546043434417916) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7290142857142858) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13381993498072356) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9295428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.046578487700934236) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4326952380952381) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07212858795272402) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7341666666666667) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10114866635456188) - val-aux/sciknoweval/score/mean@16:np.float64(0.6901785714285714) - val-aux/sciknoweval/score/std@16:np.float64(0.21497342883717385) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7771380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17605374457438094) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6025857142857143) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18719434569533389) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6898333333333334) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21523251642703473) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8483380952380953) - val-aux/sciknoweval/score/best@4/std:np.float64(0.128790863116223) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5304333333333333) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14423738321883778) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7146857142857143) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17575431812173817) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8984857142857143) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08310585797508012) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47553809523809526) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10546043434417916) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7290142857142858) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13381993498072356) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9295428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.046578487700934236) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4326952380952381) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07212858795272402) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7341666666666667) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10114866635456188) - val-core/sciknoweval/acc/mean@16:np.float64(0.6901785714285714) - val-aux/sciknoweval/acc/std@16:np.float64(0.21497342883717385) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7771380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17605374457438094) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6025857142857143) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18719434569533389) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6898333333333334) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21523251642703473) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8483380952380953) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.128790863116223) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5304333333333333) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14423738321883778) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7146857142857143) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17575431812173817) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8984857142857143) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08310585797508012) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47553809523809526) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10546043434417916) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7290142857142858) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13381993498072356) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9295428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.046578487700934236) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4326952380952381) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07212858795272402) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7341666666666667) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10114866635456188) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9645833333333333) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08812713073175192) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9939428571428572) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.03432708478331659) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9356666666666668) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11103403939640785) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9648952380952381) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08777577522757704) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996333333333334) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004692755954297982) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.890790476190476) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12713323596686327) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9825095238095238) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.06141628384478388) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8332238095238094) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.12414222258147589) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9934761904761904) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03401010083743055) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7765571428571428) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0965096914106855) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9985285714285715) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.011901839678247213) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025611198507249355 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025611198507249355 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:832.765625 - response_length/max:1661.0 - response_length/min:260.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:832.765625 - response_length_non_aborted/max:1661.0 - response_length_non_aborted/min:260.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019051693379878998 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3696037754416466) - timing_s/agent_loop/generate_sequences/max:np.float64(15.491487443447113) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.964753172069322) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.491487443447113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:255 - timing_s/agent_loop/slowest/response_length:1661 - timing_s/gen:22.220765927806497 - timing_s/reward:0.08236124739050865 - timing_s/old_log_prob:3.2945325896143913 - timing_s/adv:0.11797017231583595 - timing_s/update_actor:13.748360374942422 - timing_s/step:39.5498822350055 - timing_s/testing:247.1615258976817 - timing_s/save_checkpoint:12.889757573604584 - timing_s/stop_profile:6.954558193683624e-05 - timing_per_token_ms/adv:0.00040819563852346666 - timing_per_token_ms/gen:0.1042308475514874 - timing_per_token_ms/update_actor:0.047571522798793175 - perf/total_num_tokens:289004 - perf/time_per_step:39.5498822350055 - perf/throughput:913.4161205674948 (TaskRunner pid=2224264) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2224264) "np.float64(0.6901785714285714), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2224264) "np.float64(0.21497342883717385), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2224264) "np.float64(0.7771380952380953), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2224264) "np.float64(0.17605374457438094), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2224264) "np.float64(0.6025857142857143), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2224264) "np.float64(0.18719434569533389), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2224264) "np.float64(0.6898333333333334), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2224264) "np.float64(0.21523251642703473), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2224264) "np.float64(0.8483380952380953), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2224264) "np.float64(0.128790863116223), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2224264) "np.float64(0.5304333333333333), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2224264) "np.float64(0.14423738321883778), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2224264) "np.float64(0.7146857142857143), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2224264) "np.float64(0.17575431812173817), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2224264) "np.float64(0.8984857142857143), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2224264) "np.float64(0.08310585797508012), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2224264) "np.float64(0.47553809523809526), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2224264) "np.float64(0.10546043434417916), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2224264) "np.float64(0.7290142857142858), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2224264) "np.float64(0.13381993498072356), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2224264) "np.float64(0.9295428571428572), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2224264) 'np.float64(0.046578487700934236), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.4326952380952381), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.07212858795272402), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7341666666666667), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.10114866635456188), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6901785714285714), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/std@16': np.float64(0.21497342883717385), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7771380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.17605374457438094), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6025857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.18719434569533389), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6898333333333334), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.21523251642703473), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.8483380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.128790863116223), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5304333333333333), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.14423738321883778), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7146857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.17575431812173817), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8984857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.08310585797508012), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.47553809523809526), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.10546043434417916), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7290142857142858), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.13381993498072356), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.9295428571428572), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.046578487700934236), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.4326952380952381), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.07212858795272402), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7341666666666667), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.10114866635456188), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6901785714285714), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/std@16': np.float64(0.21497342883717385), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7771380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.17605374457438094), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6025857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.18719434569533389), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6898333333333334), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.21523251642703473), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.8483380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.128790863116223), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5304333333333333), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.14423738321883778), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7146857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.17575431812173817), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8984857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.08310585797508012), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.47553809523809526), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.10546043434417916), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7290142857142858), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.13381993498072356), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.9295428571428572), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.046578487700934236), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.4326952380952381), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.07212858795272402), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7341666666666667), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.10114866635456188), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2224264) 'np.float64(0.9645833333333333), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2224264) 'np.float64(0.08812713073175192), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2224264) 'np.float64(0.9939428571428572), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2224264) 'np.float64(0.03432708478331659), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2224264) 'np.float64(0.9356666666666668), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2224264) 'np.float64(0.11103403939640785), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': " (TaskRunner pid=2224264) 'np.float64(0.9648952380952381), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2224264) 'np.float64(0.08777577522757704), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2224264) 'np.float64(0.9996333333333334), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2224264) 'np.float64(0.004692755954297982), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2224264) 'np.float64(0.890790476190476), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2224264) 'np.float64(0.12713323596686327), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2224264) 'np.float64(0.9825095238095238), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2224264) 'np.float64(0.06141628384478388), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2224264) 'np.float64(0.8332238095238094), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2224264) 'np.float64(0.12414222258147589), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2224264) 'np.float64(0.9934761904761904), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2224264) 'np.float64(0.03401010083743055), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2224264) 'np.float64(0.7765571428571428), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2224264) 'np.float64(0.0965096914106855), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2224264) 'np.float64(0.9985285714285715), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2224264) "np.float64(0.011901839678247213), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2224264) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2224264) 'np.float64(2.0)}') (TaskRunner pid=2224264) Training Progress: 100%|██████████| 100/100 [1:42:46<00:00, 120.29s/it] Training Progress: 100%|██████████| 100/100 [1:42:46<00:00, 61.66s/it] (TaskRunner pid=2224264) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2224264) wandb: uploading output.log (TaskRunner pid=2224264) wandb: uploading history steps 98-99, summary, console lines 1082-1211 (TaskRunner pid=2224264) wandb: (TaskRunner pid=2224264) wandb: Run history: (TaskRunner pid=2224264) wandb: actor/entropy ▁▁▂▃▄▄▅▅▅▅▅▅▅▆▆▆▆▆▆▄▆▅▇▇▆▆█▆▆▆▆▆▆▇▆▅▅▆▅▆ (TaskRunner pid=2224264) wandb: actor/grad_norm █▇▇█▅▅▆▇▅▆▅▄▅▆▄▄▅▄▃▄▃▄▃▂▃▅▄▃▂▂▂▃▁▃▁▂▁▃▂▂ (TaskRunner pid=2224264) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2224264) wandb: actor/lr ▁▃▆█████████████████████████████████████ (TaskRunner pid=2224264) wandb: actor/pg_clipfrac ▇▃██▆▆▆▄▃▄▃▅▅▄▅▆▆▄▅▄▃▅▄▇▃▇▆▃▃▁▄▃▄▆▃▆▄▃▂▃ (TaskRunner pid=2224264) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▅▁▁▁▁▁▁▁▁▁▅▁▁▁▁█▁▁▁▁ (TaskRunner pid=2224264) wandb: actor/pg_loss ▂▂▅▆▂▃▃█▂▄▃▁▄▆▅▄▇▆▄▅▃▆▄▃▄▂▅▄▄▄▅▅▅▆▃▄▅▄▄▄ (TaskRunner pid=2224264) wandb: actor/ppo_kl ▆▆▁▄▄▄▆▇▅▄▇▅▃▆▇▅▄▄█▆▃▃▄▅▅▄▇▅▁▄▅▆▇▄▄▃▂▇▄▅ (TaskRunner pid=2224264) wandb: critic/advantages/max ███████████████████████████████▁████████ (TaskRunner pid=2224264) wandb: critic/advantages/mean ▃▄▄▆▄▇▆▄▆▄▃▂▁▃▄▁▄▄▅▅▄▂▂▅▄▃▄█▆▇▇▆▇▃▅▄▃▁▇▆ (TaskRunner pid=2224264) wandb: +204 ... (TaskRunner pid=2224264) wandb: (TaskRunner pid=2224264) wandb: Run summary: (TaskRunner pid=2224264) wandb: actor/entropy 0.75331 (TaskRunner pid=2224264) wandb: actor/grad_norm 0.20594 (TaskRunner pid=2224264) wandb: actor/kl_loss 0 (TaskRunner pid=2224264) wandb: actor/lr 0.0 (TaskRunner pid=2224264) wandb: actor/pg_clipfrac 0.00016 (TaskRunner pid=2224264) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2224264) wandb: actor/pg_loss 2e-05 (TaskRunner pid=2224264) wandb: actor/ppo_kl 5e-05 (TaskRunner pid=2224264) wandb: critic/advantages/max 0.875 (TaskRunner pid=2224264) wandb: critic/advantages/mean 0.00256 (TaskRunner pid=2224264) wandb: +204 ... (TaskRunner pid=2224264) wandb: (TaskRunner pid=2224264) wandb: 🚀 View run qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/ffovp19e (TaskRunner pid=2224264) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2224264) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2224264) wandb: Find logs at: ./wandb/run-20260702_235814-ffovp19e/logs (TaskRunner pid=2224264) Exception ignored in atexit callback: .teardown_atexit at 0x7797755e3880> (TaskRunner pid=2224264) Traceback (most recent call last): (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2224264) conn.teardown(hooks.exit_code) (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2224264) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2224264) return future.result() (TaskRunner pid=2224264) ^^^^^^^^^^^^^^^ (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2224264) return self.__get_result() (TaskRunner pid=2224264) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2224264) raise self._exception (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2224264) return await fn() (TaskRunner pid=2224264) ^^^^^^^^^^ (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2224264) await self._client.close() (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2224264) await self._writer.wait_closed() (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2224264) await self._protocol._get_close_waiter(self) (TaskRunner pid=2224264) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 01:41:07] Finished Qwen/Qwen3-8B chemistry grpo. [2026-07-03 01:41:07] Starting Qwen/Qwen3-8B chemistry rlsd_tr. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B 2026-07-03 01:41:09,351 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 01:41:19,207 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2255397) TaskRunner hostname: mole-workspace-rw, PID: 2255397 (TaskRunner pid=2255397) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2255397) 'calculate_entropy': False, (TaskRunner pid=2255397) 'calculate_sum_pi_squared': False, (TaskRunner pid=2255397) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2255397) 'async_save': False, (TaskRunner pid=2255397) 'load_contents': ['model', (TaskRunner pid=2255397) 'optimizer', (TaskRunner pid=2255397) 'extra'], (TaskRunner pid=2255397) 'save_contents': ['model', (TaskRunner pid=2255397) 'optimizer', (TaskRunner pid=2255397) 'extra']}, (TaskRunner pid=2255397) 'clip_ratio': 0.2, (TaskRunner pid=2255397) 'clip_ratio_c': 3.0, (TaskRunner pid=2255397) 'clip_ratio_high': 0.28, (TaskRunner pid=2255397) 'clip_ratio_low': 0.2, (TaskRunner pid=2255397) 'data_loader_seed': 42, (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_coeff': 0, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'freeze_vision_tower': False, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'forward_only': False, (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'full_determinism': False, (TaskRunner pid=2255397) 'model_dtype': 'fp32', (TaskRunner pid=2255397) 'offload_policy': False, (TaskRunner pid=2255397) 'optimizer_offload': False, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'seed': 42, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_orig_params': False, (TaskRunner pid=2255397) 'use_torch_compile': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'grad_clip': 1.0, (TaskRunner pid=2255397) 'kl_loss_coef': 0.001, (TaskRunner pid=2255397) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2255397) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2255397) 'loss_scale_factor': None, (TaskRunner pid=2255397) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2255397) 'betas': [0.9, 0.999], (TaskRunner pid=2255397) 'clip_grad': 1.0, (TaskRunner pid=2255397) 'lr': 1e-06, (TaskRunner pid=2255397) 'lr_scheduler_type': 'constant', (TaskRunner pid=2255397) 'lr_warmup_steps': 10, (TaskRunner pid=2255397) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2255397) 'min_lr_ratio': 0.0, (TaskRunner pid=2255397) 'num_cycles': 0.5, (TaskRunner pid=2255397) 'optimizer': 'AdamW', (TaskRunner pid=2255397) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2255397) 'override_optimizer_config': None, (TaskRunner pid=2255397) 'total_training_steps': -1, (TaskRunner pid=2255397) 'warmup_style': None, (TaskRunner pid=2255397) 'weight_decay': 0.01}, (TaskRunner pid=2255397) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2255397) 'clip_cov_lb': 1.0, (TaskRunner pid=2255397) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2255397) 'clip_cov_ub': 5.0, (TaskRunner pid=2255397) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2255397) 'loss_mode': 'rlsd', (TaskRunner pid=2255397) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2255397) 'ppo_epochs': 1, (TaskRunner pid=2255397) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2255397) 'ppo_micro_batch_size': None, (TaskRunner pid=2255397) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2255397) 'ppo_mini_batch_size': 8, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'rollout_n': 8, (TaskRunner pid=2255397) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2255397) 'mode': 'disabled', (TaskRunner pid=2255397) 'record_file': None, (TaskRunner pid=2255397) 'replay_file': None}, (TaskRunner pid=2255397) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2255397) 'alpha': 0.5, (TaskRunner pid=2255397) 'distillation_add_tail': True, (TaskRunner pid=2255397) 'distillation_topk': 100, (TaskRunner pid=2255397) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2255397) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2255397) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'loss_weight': 0.0, (TaskRunner pid=2255397) 'mask': 'all'}, (TaskRunner pid=2255397) 'feedback_template': '\n' (TaskRunner pid=2255397) 'The ' (TaskRunner pid=2255397) 'following ' (TaskRunner pid=2255397) 'is ' (TaskRunner pid=2255397) 'feedback ' (TaskRunner pid=2255397) 'from ' (TaskRunner pid=2255397) 'your ' (TaskRunner pid=2255397) 'unsuccessful ' (TaskRunner pid=2255397) 'earlier ' (TaskRunner pid=2255397) 'attempt:\n' (TaskRunner pid=2255397) '\n' (TaskRunner pid=2255397) '{feedback_raw}', (TaskRunner pid=2255397) 'full_logit_distillation': True, (TaskRunner pid=2255397) 'include_environment_feedback': True, (TaskRunner pid=2255397) 'is_clip': 2, (TaskRunner pid=2255397) 'max_reprompt_len': 10240, (TaskRunner pid=2255397) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2255397) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2255397) '\n' (TaskRunner pid=2255397) 'Correctly ' (TaskRunner pid=2255397) 'solve ' (TaskRunner pid=2255397) 'the ' (TaskRunner pid=2255397) 'original ' (TaskRunner pid=2255397) 'question.', (TaskRunner pid=2255397) 'reprompt_truncation': 'right', (TaskRunner pid=2255397) 'solution_template': '\n' (TaskRunner pid=2255397) 'Correct ' (TaskRunner pid=2255397) 'solution:\n' (TaskRunner pid=2255397) '\n' (TaskRunner pid=2255397) '{successful_previous_attempt}', (TaskRunner pid=2255397) 'srpo_dynamic_weighting': False, (TaskRunner pid=2255397) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2255397) 'success_reward_threshold': 0.5, (TaskRunner pid=2255397) 'teacher_regularization': 'trust-region', (TaskRunner pid=2255397) 'teacher_update_rate': 0.1, (TaskRunner pid=2255397) 'token_reweight_decay_steps': 0, (TaskRunner pid=2255397) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2255397) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2255397) 'shuffle': False, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2255397) 'tau_neg': 1.05, (TaskRunner pid=2255397) 'tau_pos': 1.0, (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_dynamic_bsz': False, (TaskRunner pid=2255397) 'use_fused_kernels': False, (TaskRunner pid=2255397) 'use_kl_loss': False, (TaskRunner pid=2255397) 'use_prefix_grouper': False, (TaskRunner pid=2255397) 'use_remove_padding': True, (TaskRunner pid=2255397) 'use_torch_compile': True}, (TaskRunner pid=2255397) 'hybrid_engine': True, (TaskRunner pid=2255397) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2255397) 'custom_chat_template': None, (TaskRunner pid=2255397) 'enable_activation_offload': False, (TaskRunner pid=2255397) 'enable_gradient_checkpointing': True, (TaskRunner pid=2255397) 'exclude_modules': None, (TaskRunner pid=2255397) 'external_lib': None, (TaskRunner pid=2255397) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2255397) 'hf_config_path': None, (TaskRunner pid=2255397) 'lora_adapter_path': None, (TaskRunner pid=2255397) 'lora_alpha': 16, (TaskRunner pid=2255397) 'lora_rank': 0, (TaskRunner pid=2255397) 'override_config': {}, (TaskRunner pid=2255397) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'target_modules': 'all-linear', (TaskRunner pid=2255397) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2255397) 'num_shards': 4}, (TaskRunner pid=2255397) 'tokenizer_path': None, (TaskRunner pid=2255397) 'trust_remote_code': True, (TaskRunner pid=2255397) 'use_fused_kernels': False, (TaskRunner pid=2255397) 'use_liger': False, (TaskRunner pid=2255397) 'use_remove_padding': True, (TaskRunner pid=2255397) 'use_shm': False}, (TaskRunner pid=2255397) 'nccl_timeout': 600, (TaskRunner pid=2255397) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'forward_only': True, (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'full_determinism': False, (TaskRunner pid=2255397) 'model_dtype': 'fp32', (TaskRunner pid=2255397) 'offload_policy': False, (TaskRunner pid=2255397) 'optimizer_offload': False, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'seed': 42, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_orig_params': False, (TaskRunner pid=2255397) 'use_torch_compile': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2255397) 'log_prob_micro_batch_size': None, (TaskRunner pid=2255397) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2255397) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'rollout_n': 8, (TaskRunner pid=2255397) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2255397) 'mode': 'disabled', (TaskRunner pid=2255397) 'record_file': None, (TaskRunner pid=2255397) 'replay_file': None}, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_torch_compile': True}, (TaskRunner pid=2255397) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2255397) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2255397) 'agent_loop_config_path': None, (TaskRunner pid=2255397) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2255397) 'name': None, (TaskRunner pid=2255397) 'path': None}, (TaskRunner pid=2255397) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2255397) 'num_workers': 8}, (TaskRunner pid=2255397) 'calculate_log_probs': True, (TaskRunner pid=2255397) 'cudagraph_capture_sizes': None, (TaskRunner pid=2255397) 'data_parallel_size': 1, (TaskRunner pid=2255397) 'disable_log_stats': True, (TaskRunner pid=2255397) 'do_sample': True, (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'enable_chunked_prefill': True, (TaskRunner pid=2255397) 'enable_prefix_caching': True, (TaskRunner pid=2255397) 'enable_rollout_routing_replay': False, (TaskRunner pid=2255397) 'enforce_eager': False, (TaskRunner pid=2255397) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2255397) 'expert_parallel_size': 1, (TaskRunner pid=2255397) 'free_cache_engine': True, (TaskRunner pid=2255397) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2255397) 'ignore_eos': False, (TaskRunner pid=2255397) 'layered_summon': False, (TaskRunner pid=2255397) 'load_format': 'dummy', (TaskRunner pid=2255397) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2255397) 'log_prob_micro_batch_size': None, (TaskRunner pid=2255397) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2255397) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2255397) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2255397) 'max_model_len': 10240, (TaskRunner pid=2255397) 'max_num_batched_tokens': 10240, (TaskRunner pid=2255397) 'max_num_seqs': 1024, (TaskRunner pid=2255397) 'mode': 'async', (TaskRunner pid=2255397) 'multi_stage_wake_up': False, (TaskRunner pid=2255397) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'format': 'hermes', (TaskRunner pid=2255397) 'interaction_config_path': None, (TaskRunner pid=2255397) 'max_assistant_turns': None, (TaskRunner pid=2255397) 'max_parallel_calls': 1, (TaskRunner pid=2255397) 'max_tool_response_length': 256, (TaskRunner pid=2255397) 'max_user_turns': None, (TaskRunner pid=2255397) 'num_repeat_rollouts': None, (TaskRunner pid=2255397) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2255397) 'tool_config_path': None, (TaskRunner pid=2255397) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2255397) 'use_inference_chat_template': False}, (TaskRunner pid=2255397) 'n': 8, (TaskRunner pid=2255397) 'name': 'vllm', (TaskRunner pid=2255397) 'over_sample_rate': 0, (TaskRunner pid=2255397) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2255397) 'port': 9090, (TaskRunner pid=2255397) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2255397) 'prompt_length': 2048, (TaskRunner pid=2255397) 'quantization': None, (TaskRunner pid=2255397) 'quantization_config_file': None, (TaskRunner pid=2255397) 'response_length': 8192, (TaskRunner pid=2255397) 'scheduling_policy': 'fcfs', (TaskRunner pid=2255397) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2255397) 'skip_rollout': False, (TaskRunner pid=2255397) 'skip_tokenizer_init': True, (TaskRunner pid=2255397) 'temperature': 1.0, (TaskRunner pid=2255397) 'tensor_model_parallel_size': 2, (TaskRunner pid=2255397) 'top_k': -1, (TaskRunner pid=2255397) 'top_p': 1.0, (TaskRunner pid=2255397) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2255397) 'backend': None, (TaskRunner pid=2255397) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2255397) 'token2text': False}, (TaskRunner pid=2255397) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2255397) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2255397) 'do_sample': True, (TaskRunner pid=2255397) 'n': 16, (TaskRunner pid=2255397) 'temperature': 0.6, (TaskRunner pid=2255397) 'top_k': -1, (TaskRunner pid=2255397) 'top_p': 0.95}}}, (TaskRunner pid=2255397) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2255397) 'adv_estimator': 'grpo', (TaskRunner pid=2255397) 'gamma': 1.0, (TaskRunner pid=2255397) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2255397) 'horizon': 10000, (TaskRunner pid=2255397) 'kl_coef': 0.001, (TaskRunner pid=2255397) 'target_kl': 0.1, (TaskRunner pid=2255397) 'type': 'fixed'}, (TaskRunner pid=2255397) 'kl_penalty': 'kl', (TaskRunner pid=2255397) 'lam': 1.0, (TaskRunner pid=2255397) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2255397) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2255397) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2255397) 'loss_type': 'ppo_clip', (TaskRunner pid=2255397) 'rollout_is': 'token', (TaskRunner pid=2255397) 'rollout_is_batch_normalize': False, (TaskRunner pid=2255397) 'rollout_is_threshold': 2.0, (TaskRunner pid=2255397) 'rollout_rs': None, (TaskRunner pid=2255397) 'rollout_rs_threshold': None}, (TaskRunner pid=2255397) 'use_kl_in_reward': False, (TaskRunner pid=2255397) 'use_pf_ppo': False}, (TaskRunner pid=2255397) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2255397) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2255397) 'async_save': False, (TaskRunner pid=2255397) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2255397) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2255397) 'cliprange_value': 0.5, (TaskRunner pid=2255397) 'data_loader_seed': 42, (TaskRunner pid=2255397) 'enable': None, (TaskRunner pid=2255397) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2255397) 'forward_micro_batch_size': None, (TaskRunner pid=2255397) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2255397) 'grad_clip': 1.0, (TaskRunner pid=2255397) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2255397) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2255397) 'enable_activation_offload': False, (TaskRunner pid=2255397) 'enable_gradient_checkpointing': True, (TaskRunner pid=2255397) 'external_lib': None, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'forward_only': False, (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'full_determinism': False, (TaskRunner pid=2255397) 'model_dtype': 'fp32', (TaskRunner pid=2255397) 'offload_policy': False, (TaskRunner pid=2255397) 'optimizer_offload': False, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'seed': 42, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_orig_params': False, (TaskRunner pid=2255397) 'use_torch_compile': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'lora_alpha': 16, (TaskRunner pid=2255397) 'lora_rank': 0, (TaskRunner pid=2255397) 'override_config': {}, (TaskRunner pid=2255397) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'target_modules': 'all-linear', (TaskRunner pid=2255397) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2255397) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'trust_remote_code': True, (TaskRunner pid=2255397) 'use_remove_padding': False, (TaskRunner pid=2255397) 'use_shm': False}, (TaskRunner pid=2255397) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2255397) 'betas': [0.9, 0.999], (TaskRunner pid=2255397) 'clip_grad': 1.0, (TaskRunner pid=2255397) 'lr': 1e-05, (TaskRunner pid=2255397) 'lr_scheduler_type': 'constant', (TaskRunner pid=2255397) 'lr_warmup_steps': -1, (TaskRunner pid=2255397) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2255397) 'min_lr_ratio': 0.0, (TaskRunner pid=2255397) 'num_cycles': 0.5, (TaskRunner pid=2255397) 'optimizer': 'AdamW', (TaskRunner pid=2255397) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2255397) 'override_optimizer_config': None, (TaskRunner pid=2255397) 'total_training_steps': -1, (TaskRunner pid=2255397) 'warmup_style': None, (TaskRunner pid=2255397) 'weight_decay': 0.01}, (TaskRunner pid=2255397) 'ppo_epochs': 1, (TaskRunner pid=2255397) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2255397) 'ppo_micro_batch_size': None, (TaskRunner pid=2255397) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2255397) 'ppo_mini_batch_size': 8, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'rollout_n': 8, (TaskRunner pid=2255397) 'shuffle': False, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_dynamic_bsz': False}, (TaskRunner pid=2255397) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2255397) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2255397) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2255397) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2255397) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2255397) 'dataloader_num_workers': 8, (TaskRunner pid=2255397) 'filter_overlong_prompts': True, (TaskRunner pid=2255397) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2255397) 'image_key': 'images', (TaskRunner pid=2255397) 'image_patch_size': 14, (TaskRunner pid=2255397) 'max_prompt_length': 2048, (TaskRunner pid=2255397) 'max_response_length': 8192, (TaskRunner pid=2255397) 'prompt_key': 'prompt', (TaskRunner pid=2255397) 'return_full_prompt': False, (TaskRunner pid=2255397) 'return_multi_modal_inputs': True, (TaskRunner pid=2255397) 'return_raw_chat': True, (TaskRunner pid=2255397) 'return_raw_input_ids': False, (TaskRunner pid=2255397) 'reward_fn_key': 'data_source', (TaskRunner pid=2255397) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2255397) 'seed': None, (TaskRunner pid=2255397) 'shuffle': True, (TaskRunner pid=2255397) 'tokenizer': None, (TaskRunner pid=2255397) 'tool_config_path': None, (TaskRunner pid=2255397) 'train_batch_size': 32, (TaskRunner pid=2255397) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2255397) 'train_max_samples': 3200, (TaskRunner pid=2255397) 'truncation': 'error', (TaskRunner pid=2255397) 'trust_remote_code': True, (TaskRunner pid=2255397) 'use_shm': False, (TaskRunner pid=2255397) 'val_batch_size': None, (TaskRunner pid=2255397) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2255397) 'val_max_samples': -1, (TaskRunner pid=2255397) 'validation_shuffle': False, (TaskRunner pid=2255397) 'video_key': 'videos'}, (TaskRunner pid=2255397) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2255397) 'cuda-memory-usage': 'true', (TaskRunner pid=2255397) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2255397) 'capture-range-end': None, (TaskRunner pid=2255397) 'cuda-graph-trace': 'graph', (TaskRunner pid=2255397) 'cuda-memory-usage': 'true', (TaskRunner pid=2255397) 'kill': 'none', (TaskRunner pid=2255397) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2255397) 'torch_memory': {'context': 'all', (TaskRunner pid=2255397) 'kw_args': {}, (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'stacks': 'all', (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2255397) 'profile_continuous_steps': False, (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'steps': None, (TaskRunner pid=2255397) 'tool': None}, (TaskRunner pid=2255397) 'max_model_len': 10240, (TaskRunner pid=2255397) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B', (TaskRunner pid=2255397) 'include_dashboard': False, (TaskRunner pid=2255397) 'num_cpus': None, (TaskRunner pid=2255397) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2255397) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2255397) 'USER': 'root'}}}, (TaskRunner pid=2255397) 'timeline_json_file': None}, (TaskRunner pid=2255397) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2255397) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2255397) 'name': 'custom_reward_manager', (TaskRunner pid=2255397) 'path': None}, (TaskRunner pid=2255397) 'name': 'naive', (TaskRunner pid=2255397) 'source': 'register'}, (TaskRunner pid=2255397) 'reward_model': {'enable': False, (TaskRunner pid=2255397) 'enable_resource_pool': False, (TaskRunner pid=2255397) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2255397) 'launch_reward_fn_async': False, (TaskRunner pid=2255397) 'max_length': None, (TaskRunner pid=2255397) 'micro_batch_size': None, (TaskRunner pid=2255397) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2255397) 'model': {'external_lib': None, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'override_config': {}, (TaskRunner pid=2255397) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2255397) 'trust_remote_code': False, (TaskRunner pid=2255397) 'use_fused_kernels': False, (TaskRunner pid=2255397) 'use_remove_padding': False, (TaskRunner pid=2255397) 'use_shm': False}, (TaskRunner pid=2255397) 'n_gpus_per_node': 8, (TaskRunner pid=2255397) 'nnodes': 0, (TaskRunner pid=2255397) 'num_workers': 1, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'reward_loop_class_name': None, (TaskRunner pid=2255397) 'reward_loop_module_path': None, (TaskRunner pid=2255397) 'reward_loop_source': 'register', (TaskRunner pid=2255397) 'reward_manager': 'naive', (TaskRunner pid=2255397) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2255397) 'cudagraph_capture_sizes': None, (TaskRunner pid=2255397) 'data_parallel_size': 1, (TaskRunner pid=2255397) 'disable_log_stats': True, (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'enable_chunked_prefill': True, (TaskRunner pid=2255397) 'enable_prefix_caching': True, (TaskRunner pid=2255397) 'enforce_eager': True, (TaskRunner pid=2255397) 'engine_kwargs': {}, (TaskRunner pid=2255397) 'expert_parallel_size': 1, (TaskRunner pid=2255397) 'free_cache_engine': True, (TaskRunner pid=2255397) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2255397) 'limit_images': None, (TaskRunner pid=2255397) 'load_format': 'auto', (TaskRunner pid=2255397) 'max_model_len': None, (TaskRunner pid=2255397) 'max_num_batched_tokens': 8192, (TaskRunner pid=2255397) 'max_num_seqs': 1024, (TaskRunner pid=2255397) 'name': '???', (TaskRunner pid=2255397) 'prompt_length': 2048, (TaskRunner pid=2255397) 'response_length': 2048, (TaskRunner pid=2255397) 'skip_tokenizer_init': False, (TaskRunner pid=2255397) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2255397) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2255397) 'memory_limit_mb': 1024, (TaskRunner pid=2255397) 'url': None}, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_dynamic_bsz': False, (TaskRunner pid=2255397) 'use_reward_loop': False}, (TaskRunner pid=2255397) 'trainer': {'balance_batch': True, (TaskRunner pid=2255397) 'critic_warmup': 0, (TaskRunner pid=2255397) 'default_hdfs_dir': None, (TaskRunner pid=2255397) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2255397) 'del_local_ckpt_after_load': False, (TaskRunner pid=2255397) 'device': 'cuda', (TaskRunner pid=2255397) 'esi_redundant_time': 0, (TaskRunner pid=2255397) 'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2255397) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2255397) 'log_val_generations': 0, (TaskRunner pid=2255397) 'logger': ['console', 'wandb'], (TaskRunner pid=2255397) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2255397) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2255397) 'n_gpus_per_node': 8, (TaskRunner pid=2255397) 'nnodes': 1, (TaskRunner pid=2255397) 'project_name': 'SDPO-root', (TaskRunner pid=2255397) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2255397) 'resume_from_path': None, (TaskRunner pid=2255397) 'resume_mode': 'auto', (TaskRunner pid=2255397) 'rollout_data_dir': None, (TaskRunner pid=2255397) 'save_freq': 10, (TaskRunner pid=2255397) 'test_freq': 10, (TaskRunner pid=2255397) 'total_epochs': 30, (TaskRunner pid=2255397) 'total_training_steps': 100, (TaskRunner pid=2255397) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2255397) 'val_before_train': False, (TaskRunner pid=2255397) 'val_only': False, (TaskRunner pid=2255397) 'validation_data_dir': None}, (TaskRunner pid=2255397) 'transfer_queue': {'enable': False}, (TaskRunner pid=2255397) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2255397) 'dir': '/users/root/SDPO', (TaskRunner pid=2255397) 'log_dir': '/users/root/output', (TaskRunner pid=2255397) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2255397) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2255397) use_critic=need_critic(config), (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2255397) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2255397) Using dataset class: RLHFDataset (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2255397) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2255397) dataset len: 1890 (TaskRunner pid=2255397) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2255397) WARNING:2026-07-03 01:41:28,094:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2255397) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2255397) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2255397) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 276.01 examples/s] (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2255397) self.use_critic = need_critic(self.config) (WorkerDict pid=2255793) [W703 01:41:37.908542315 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:41933 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2255788) [Gloo] Rank 1 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2255789) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2255789) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2255792) [W703 01:41:38.533011673 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:41933 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2255787) Model config after override: Qwen3Config { (WorkerDict pid=2255787) "architectures": [ (WorkerDict pid=2255787) "Qwen3ForCausalLM" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "attention_bias": false, (WorkerDict pid=2255787) "attention_dropout": 0.0, (WorkerDict pid=2255787) "dtype": "bfloat16", (WorkerDict pid=2255787) "eos_token_id": 151645, (WorkerDict pid=2255787) "head_dim": 128, (WorkerDict pid=2255787) "hidden_act": "silu", (WorkerDict pid=2255787) "hidden_size": 4096, (WorkerDict pid=2255787) "initializer_range": 0.02, (WorkerDict pid=2255787) "intermediate_size": 12288, (WorkerDict pid=2255787) "layer_types": [ (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "max_position_embeddings": 40960, (WorkerDict pid=2255787) "max_window_layers": 36, (WorkerDict pid=2255787) "model_type": "qwen3", (WorkerDict pid=2255787) "num_attention_heads": 32, (WorkerDict pid=2255787) "num_hidden_layers": 36, (WorkerDict pid=2255787) "num_key_value_heads": 8, (WorkerDict pid=2255787) "pad_token_id": 151643, (WorkerDict pid=2255787) "rms_norm_eps": 1e-06, (WorkerDict pid=2255787) "rope_scaling": null, (WorkerDict pid=2255787) "rope_theta": 1000000, (WorkerDict pid=2255787) "sliding_window": null, (WorkerDict pid=2255787) "tie_word_embeddings": false, (WorkerDict pid=2255787) "transformers_version": "4.57.1", (WorkerDict pid=2255787) "use_cache": true, (WorkerDict pid=2255787) "use_sliding_window": false, (WorkerDict pid=2255787) "vocab_size": 151936 (WorkerDict pid=2255787) } (WorkerDict pid=2255787) (WorkerDict pid=2255794) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2255794) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2255794) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2255789) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2255787) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2255788) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2255788) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2255790) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.90s/it] [repeated 15x across cluster] (WorkerDict pid=2255790) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.20s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.60s/it] [repeated 7x across cluster] (WorkerDict pid=2255787) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2255790) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2255790) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2255787) Actor use_remove_padding=True (WorkerDict pid=2255787) Actor use_fused_kernels=False (WorkerDict pid=2255787) Actor use_prefix_grouper=False (WorkerDict pid=2255794) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2255794) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2255788) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2255787) reference model: Qwen/Qwen3-8B (WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2255787) warnings.warn( (WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2255787) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2255787) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2255787) Model config after override: Qwen3Config { (WorkerDict pid=2255787) "architectures": [ (WorkerDict pid=2255787) "Qwen3ForCausalLM" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "attention_bias": false, (WorkerDict pid=2255787) "attention_dropout": 0.0, (WorkerDict pid=2255787) "dtype": "bfloat16", (WorkerDict pid=2255787) "eos_token_id": 151645, (WorkerDict pid=2255787) "head_dim": 128, (WorkerDict pid=2255787) "hidden_act": "silu", (WorkerDict pid=2255787) "hidden_size": 4096, (WorkerDict pid=2255787) "initializer_range": 0.02, (WorkerDict pid=2255787) "intermediate_size": 12288, (WorkerDict pid=2255787) "layer_types": [ (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "max_position_embeddings": 40960, (WorkerDict pid=2255787) "max_window_layers": 36, (WorkerDict pid=2255787) "model_type": "qwen3", (WorkerDict pid=2255787) "num_attention_heads": 32, (WorkerDict pid=2255787) "num_hidden_layers": 36, (WorkerDict pid=2255787) "num_key_value_heads": 8, (WorkerDict pid=2255787) "pad_token_id": 151643, (WorkerDict pid=2255787) "rms_norm_eps": 1e-06, (WorkerDict pid=2255787) "rope_scaling": null, (WorkerDict pid=2255787) "rope_theta": 1000000, (WorkerDict pid=2255787) "sliding_window": null, (WorkerDict pid=2255787) "tie_word_embeddings": false, (WorkerDict pid=2255787) "transformers_version": "4.57.1", (WorkerDict pid=2255787) "use_cache": true, (WorkerDict pid=2255787) "use_sliding_window": false, (WorkerDict pid=2255787) "vocab_size": 151936 (WorkerDict pid=2255787) } (WorkerDict pid=2255787) (WorkerDict pid=2255792) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2255787) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2255787) Ref use_remove_padding=True (WorkerDict pid=2255787) Ref use_fused_kernels=False (WorkerDict pid=2255787) Ref use_prefix_grouper=False (WorkerDict pid=2255792) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2255792) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2255397) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2255792) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.93s/it] [repeated 15x across cluster] (WorkerDict pid=2255792) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.62s/it] [repeated 7x across cluster] (vLLMHttpServer pid=2256706) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2256706) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2256707) ['serve', (vLLMHttpServer pid=2256707) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2256707) '--dtype', (vLLMHttpServer pid=2256707) 'bfloat16', (vLLMHttpServer pid=2256707) '--load_format', (vLLMHttpServer pid=2256707) 'dummy', (vLLMHttpServer pid=2256707) '--trust_remote_code', (vLLMHttpServer pid=2256707) '--max_model_len', (vLLMHttpServer pid=2256707) '40960', (vLLMHttpServer pid=2256707) '--max_num_seqs', (vLLMHttpServer pid=2256707) '1024', (vLLMHttpServer pid=2256707) '--enable_chunked_prefill', (vLLMHttpServer pid=2256707) '--max_num_batched_tokens', (vLLMHttpServer pid=2256707) '10240', (vLLMHttpServer pid=2256707) '--enable_prefix_caching', (vLLMHttpServer pid=2256707) '--enable_sleep_mode', (vLLMHttpServer pid=2256707) '--logprobs_mode', (vLLMHttpServer pid=2256707) 'processed_logprobs', (vLLMHttpServer pid=2256707) '--disable_custom_all_reduce', (vLLMHttpServer pid=2256707) '--gpu_memory_utilization', (vLLMHttpServer pid=2256707) '0.8', (vLLMHttpServer pid=2256707) '--disable_log_stats', (vLLMHttpServer pid=2256707) '--tensor_parallel_size', (vLLMHttpServer pid=2256707) '2', (vLLMHttpServer pid=2256707) '--seed', (vLLMHttpServer pid=2256707) '0', (vLLMHttpServer pid=2256707) '--override_generation_config', (vLLMHttpServer pid=2256707) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2256707) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2256707) '--hf_overrides', (vLLMHttpServer pid=2256707) '{}', (vLLMHttpServer pid=2256707) '--scheduling_policy', (vLLMHttpServer pid=2256707) 'fcfs'] (vLLMHttpServer pid=2256707) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2256706) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2256706) WARNING 07-03 01:42:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2255788) WARNING 07-03 01:43:03 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2256709) WARNING 07-03 01:42:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2255789) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2255793) 2026-07-03 01:43:17,091 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (vLLMHttpServer pid=2256709) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2256709) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2256709) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2256709) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2255793) 2026-07-03 01:43:17,106 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2255787) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to examine the structure of the product and match it with the structure of the reactant provided in the options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key features: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or imidazole-like structure). (TaskRunner pid=2255397) - A substituent "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring substituted with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2255397) - The overall structure suggests that the reactant must have a similar core structure, with the necessary functional groups for the formation of the final product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Let's analyze the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" (TaskRunner pid=2255397) This structure contains a nitrogen-containing ring with the correct substituent "Cc2ccc(Cl)cc2C(F)(F)F". It also has a carbonyl group and a nitrogen atom, which could be part of a condensation or coupling reaction. This matches the structure of the product closely. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" (TaskRunner pid=2255397) This option introduces a chlorine atom directly on the nitrogen, which is not present in the product. Also, the structure seems to have a different arrangement of substituents, making it unlikely to be the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" (TaskRunner pid=2255397) This structure has a hydroxyl group attached to the benzene ring, which is not present in the product. The presence of this hydroxyl group suggests a different synthetic pathway and thus is not the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" (TaskRunner pid=2255397) This option has a different arrangement of the substituents and introduces a double bond between the carbon atoms, which is not part of the product structure. It also lacks the necessary nitrogen-containing ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion**: (TaskRunner pid=2255397) Only **Option A** contains the correct substituent "Cc2ccc(Cl)cc2C(F)(F)F" and the necessary nitrogen-containing ring structure. It is the only option that matches the structure of the product directly. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2255397) step:10 - global_seqlen/min:18995 - global_seqlen/max:25222 - global_seqlen/minmax_diff:6227 - global_seqlen/balanced_min:22810 - global_seqlen/balanced_max:22819 - global_seqlen/mean:22815.375 - actor/entropy:0.5086405277252197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48631101846694946 - training/rollout_probs_diff_mean:0.004814056679606438 - training/rollout_probs_diff_std:0.010112768970429897 - training/rollout_actor_probs_pearson_corr:0.9993922114372253 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.37109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0000672340393066 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7611369912629016e-05 - rollout_corr/rollout_is_max:1.758644938468933 - rollout_corr/rollout_is_min:0.47177526354789734 - rollout_corr/rollout_is_std:0.0358346551656723 - rollout_corr/rollout_is_eff_sample_size:0.9987177621962007 - rollout_corr/rollout_is_seq_mean:1.0001494884490967 - rollout_corr/rollout_is_seq_std:0.001915071508847177 - rollout_corr/rollout_is_seq_max:1.0051805973052979 - rollout_corr/rollout_is_seq_min:0.9944745898246765 - rollout_corr/rollout_is_seq_max_deviation:0.005525410175323486 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7283513699658215) - rollout_corr/training_log_ppl:np.float64(0.5342370558064431) - rollout_corr/kl:np.float64(0.00048592705383754264) - rollout_corr/k3_kl:np.float64(0.0007897751748942028) - rollout_corr/rollout_ppl:np.float64(1.7275344631634653) - rollout_corr/rollout_log_ppl:np.float64(0.5337511280667968) - rollout_corr/log_ppl_diff:np.float64(0.00048592773964628577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017402138910256326) - rollout_corr/log_ppl_diff_max:np.float64(0.008036613464355469) - rollout_corr/log_ppl_diff_min:np.float64(-0.007182866334915161) - rollout_corr/ppl_ratio:np.float64(1.0004885760135949) - rollout_corr/chi2_token:np.float64(0.002185102319344878) - rollout_corr/chi2_seq:np.float64(1.4425214361399412) - actor/pg_loss:np.float64(0.008299664361402392) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006863867054107686) - actor/ppo_kl:np.float64(-5.119195938618759e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37109375) - self_distillation/token_reweight_w_mean:np.float64(42394.31746227876) - self_distillation/token_reweight_w_std:np.float64(702394.3469068523) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012782558333129) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1625702439341694) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.3786138668656349) - perf/mfu/actor:np.float64(0.054599702355521036) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.04931259155273) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4217261904761905) - val-aux/sciknoweval/reward/std@16:np.float64(0.28826355015906874) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5430761904761905) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.26128001800402223) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3013761904761905) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2277461781455412) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4231666666666667) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2873520999211171) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6505000000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2040037803005508) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.21330476190476189) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15085658746075342) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43689523809523806) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.24031934791752282) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7316095238095237) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.13822877715965945) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.15793333333333331) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0882011251130811) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.444252380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.19013145421249977) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7838952380952381) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07974930858404744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.12557619047619048) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04538785689784369) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.4482333333333334) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.14374275063423717) - val-aux/sciknoweval/score/mean@16:np.float64(0.4217261904761905) - val-aux/sciknoweval/score/std@16:np.float64(0.28826355015906874) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5430761904761905) - val-aux/sciknoweval/score/best@2/std:np.float64(0.26128001800402223) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3013761904761905) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2277461781455412) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4231666666666667) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2873520999211171) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6505000000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2040037803005508) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.21330476190476189) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15085658746075342) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43689523809523806) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.24031934791752282) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7316095238095237) - val-aux/sciknoweval/score/best@8/std:np.float64(0.13822877715965945) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.15793333333333331) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0882011251130811) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.444252380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.19013145421249977) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7838952380952381) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07974930858404744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.12557619047619048) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04538785689784369) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.4482333333333334) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.14374275063423717) - val-core/sciknoweval/acc/mean@16:np.float64(0.4217261904761905) - val-aux/sciknoweval/acc/std@16:np.float64(0.28826355015906874) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5430761904761905) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.26128001800402223) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3013761904761905) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2277461781455412) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4231666666666667) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2873520999211171) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6505000000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2040037803005508) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.21330476190476189) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15085658746075342) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43689523809523806) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.24031934791752282) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7316095238095237) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.13822877715965945) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.15793333333333331) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0882011251130811) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.444252380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.19013145421249977) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7838952380952381) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07974930858404744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.12557619047619048) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04538785689784369) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.4482333333333334) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.14374275063423717) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9946428571428572) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.018556639291927544) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995666666666666) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.005201629481674079) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9896238095238097) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.024846621608394712) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9946904761904761) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.018505164380603217) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0001505093393264677) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9811428571428572) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0310421950289628) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9980904761904762) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.011107450564340775) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9676523809523809) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.034830287112445846) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995761904761905) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.004892275828419172) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9506190476190476) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0319196096808831) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999571428571428) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0009723855004127487) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003302131954114884 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003302131954114884 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:443.60546875 - response_length/max:1164.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:443.60546875 - response_length_non_aborted/max:1164.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001530386507511139 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8319511320441961) - timing_s/agent_loop/generate_sequences/max:np.float64(9.405133683234453) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.998352278737002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.405133683234453) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:1164 - timing_s/gen:16.217690873891115 - timing_s/reward:0.0643529836088419 - timing_s/old_log_prob:3.16649679094553 - timing_s/adv:0.5727238319814205 - timing_s/update_actor:21.792541556060314 - timing_s/step:41.90203979611397 - timing_s/testing:145.202635621652 - timing_s/save_checkpoint:13.197123931720853 - timing_s/stop_profile:0.00014385022222995758 - timing_per_token_ms/update_actor:0.11939613942385516 - timing_per_token_ms/gen:0.14280787645528134 - timing_per_token_ms/adv:0.003137817327029583 - perf/total_num_tokens:182523 - perf/time_per_step:41.90203979611397 - perf/throughput:544.4931824563805 (TaskRunner pid=2255397) Training Progress: 10%|█ | 10/100 [10:23<2:18:30, 92.33s/it] (TaskRunner pid=2255397) step:11 - global_seqlen/min:18238 - global_seqlen/max:25942 - global_seqlen/minmax_diff:7704 - global_seqlen/balanced_min:22325 - global_seqlen/balanced_max:22730 - global_seqlen/mean:22380.25 - actor/entropy:0.5434318780899048 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38079291582107544 - training/rollout_probs_diff_mean:0.005151212681084871 - training/rollout_probs_diff_std:0.010387294925749302 - training/rollout_actor_probs_pearson_corr:0.9993837475776672 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001193284988403 - rollout_corr/rollout_is_ratio_fraction_high:9.758572559803724e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.194515228271484 - rollout_corr/rollout_is_min:0.5521866679191589 - rollout_corr/rollout_is_std:0.03714950755238533 - rollout_corr/rollout_is_eff_sample_size:0.9986220538290654 - rollout_corr/rollout_is_seq_mean:1.0001423358917236 - rollout_corr/rollout_is_seq_std:0.0020691656973212957 - rollout_corr/rollout_is_seq_max:1.0071853399276733 - rollout_corr/rollout_is_seq_min:0.993625283241272 - rollout_corr/rollout_is_seq_max_deviation:0.00718533992767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7692446084693074) - rollout_corr/training_log_ppl:np.float64(0.5590797059703618) - rollout_corr/kl:np.float64(0.0006710044406990079) - rollout_corr/k3_kl:np.float64(0.0007906234051233696) - rollout_corr/rollout_ppl:np.float64(1.7680208389647305) - rollout_corr/rollout_log_ppl:np.float64(0.5584087027818896) - rollout_corr/log_ppl_diff:np.float64(0.0006710031884722412) - rollout_corr/log_ppl_abs_diff:np.float64(0.001697185740340501) - rollout_corr/log_ppl_diff_max:np.float64(0.006523817777633667) - rollout_corr/log_ppl_diff_min:np.float64(-0.005903065204620361) - rollout_corr/ppl_ratio:np.float64(1.0006734766066074) - rollout_corr/chi2_token:np.float64(0.0018631014972925186) - rollout_corr/chi2_seq:np.float64(1.0500562572851777) - actor/pg_loss:np.float64(0.010553930653259158) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004439412439296575) - actor/ppo_kl:np.float64(6.877072929745154e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(85283.25226895092) - self_distillation/token_reweight_w_std:np.float64(1114820.7389926258) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022825964260846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21137651632307097) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4761762246489525) - perf/mfu/actor:np.float64(0.05262875043012334) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.41241455078125) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003749731695279479 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003749731695279479 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:400.2890625 - response_length/max:1690.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:400.2890625 - response_length_non_aborted/max:1690.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012122094631195068 - timing_s/agent_loop/generate_sequences/min:np.float64(2.067027226090431) - timing_s/agent_loop/generate_sequences/max:np.float64(12.363675134256482) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.645478486927459) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.363675134256482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1690 - timing_s/gen:18.65842711366713 - timing_s/reward:0.055716294795274734 - timing_s/old_log_prob:3.1651459094136953 - timing_s/adv:0.6307941991835833 - timing_s/update_actor:22.146489353850484 - timing_s/step:44.69863370805979 - timing_s/stop_profile:0.00010724924504756927 - timing_per_token_ms/update_actor:0.1236943809488862 - timing_per_token_ms/gen:0.18207962130557145 - timing_per_token_ms/adv:0.0035231632755642994 - perf/total_num_tokens:179042 - perf/time_per_step:44.69863370805979 - perf/throughput:500.69203784107003 (TaskRunner pid=2255397) Training Progress: 11%|█ | 11/100 [11:08<1:55:21, 77.77s/it] (TaskRunner pid=2255397) step:12 - global_seqlen/min:19921 - global_seqlen/max:27357 - global_seqlen/minmax_diff:7436 - global_seqlen/balanced_min:23213 - global_seqlen/balanced_max:23297 - global_seqlen/mean:23233.875 - actor/entropy:0.5037309527397156 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16815555095672607 - training/rollout_probs_diff_mean:0.004780740477144718 - training/rollout_probs_diff_std:0.009941385127604008 - training/rollout_actor_probs_pearson_corr:0.9994201064109802 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.999710738658905 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.6984993635560386e-05 - rollout_corr/rollout_is_max:1.6297073364257812 - rollout_corr/rollout_is_min:0.3698853552341461 - rollout_corr/rollout_is_std:0.035432368516922 - rollout_corr/rollout_is_eff_sample_size:0.9987455864550007 - rollout_corr/rollout_is_seq_mean:0.9997212886810303 - rollout_corr/rollout_is_seq_std:0.0021238892804831266 - rollout_corr/rollout_is_seq_max:1.0065120458602905 - rollout_corr/rollout_is_seq_min:0.9934496879577637 - rollout_corr/rollout_is_seq_max_deviation:0.006550312042236328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7103436021134257) - rollout_corr/training_log_ppl:np.float64(0.5244237306469586) - rollout_corr/kl:np.float64(0.0010640324547948055) - rollout_corr/k3_kl:np.float64(0.0007240620033144296) - rollout_corr/rollout_ppl:np.float64(1.7084603938274086) - rollout_corr/rollout_log_ppl:np.float64(0.5233596969628707) - rollout_corr/log_ppl_diff:np.float64(0.0010640336840879172) - rollout_corr/log_ppl_abs_diff:np.float64(0.001916373468702659) - rollout_corr/log_ppl_diff_max:np.float64(0.007551968097686768) - rollout_corr/log_ppl_diff_min:np.float64(-0.006095826625823975) - rollout_corr/ppl_ratio:np.float64(1.001067083561793) - rollout_corr/chi2_token:np.float64(0.0007682361174374819) - rollout_corr/chi2_seq:np.float64(0.6293782293796539) - actor/pg_loss:np.float64(0.007974700187332928) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048827441059984267) - actor/ppo_kl:np.float64(0.00010889819034787251) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(5987.44374671299) - self_distillation/token_reweight_w_std:np.float64(110518.26587392716) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0042279572226107) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17869807308306918) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.39314158260822296) - perf/mfu/actor:np.float64(0.05490032892199064) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.42404556274414) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0070593878626823425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0070593878626823425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.96484375 - response_length/max:1546.0 - response_length/min:137.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.96484375 - response_length_non_aborted/max:1546.0 - response_length_non_aborted/min:137.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016945600509643555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.736042970791459) - timing_s/agent_loop/generate_sequences/max:np.float64(12.078167820349336) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.221707455886644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.078167820349336) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1546 - timing_s/gen:18.70925861224532 - timing_s/reward:0.06456440314650536 - timing_s/old_log_prob:3.0842963717877865 - timing_s/adv:0.5920961294323206 - timing_s/update_actor:22.019973430782557 - timing_s/step:44.561775021255016 - timing_s/stop_profile:0.0001148320734500885 - timing_per_token_ms/update_actor:0.11846911799464444 - timing_per_token_ms/gen:0.15888832037303566 - timing_per_token_ms/adv:0.0031855218373620445 - perf/total_num_tokens:185871 - perf/time_per_step:44.561775021255016 - perf/throughput:521.3857614270961 (TaskRunner pid=2255397) Training Progress: 12%|█▏ | 12/100 [11:52<1:39:16, 67.68s/it] (TaskRunner pid=2255397) step:13 - global_seqlen/min:18814 - global_seqlen/max:23875 - global_seqlen/minmax_diff:5061 - global_seqlen/balanced_min:21556 - global_seqlen/balanced_max:21562 - global_seqlen/mean:21560.5 - actor/entropy:0.540065348148346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2304113507270813 - training/rollout_probs_diff_mean:0.005129465367645025 - training/rollout_probs_diff_std:0.010314878076314926 - training/rollout_actor_probs_pearson_corr:0.9993942975997925 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:0.9999523758888245 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.84678445092868e-06 - rollout_corr/rollout_is_max:1.643043875694275 - rollout_corr/rollout_is_min:0.3293769061565399 - rollout_corr/rollout_is_std:0.03673160448670387 - rollout_corr/rollout_is_eff_sample_size:0.9986523694012616 - rollout_corr/rollout_is_seq_mean:0.9999930262565613 - rollout_corr/rollout_is_seq_std:0.0019651646725833416 - rollout_corr/rollout_is_seq_max:1.0069950819015503 - rollout_corr/rollout_is_seq_min:0.9950668215751648 - rollout_corr/rollout_is_seq_max_deviation:0.006995081901550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.75340425549075) - rollout_corr/training_log_ppl:np.float64(0.5529010602040216) - rollout_corr/kl:np.float64(0.0008567688251606143) - rollout_corr/k3_kl:np.float64(0.0008690143870353495) - rollout_corr/rollout_ppl:np.float64(1.7518776911310852) - rollout_corr/rollout_log_ppl:np.float64(0.5520442884298973) - rollout_corr/log_ppl_diff:np.float64(0.0008567717741243541) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018868366605602205) - rollout_corr/log_ppl_diff_max:np.float64(0.00990372896194458) - rollout_corr/log_ppl_diff_min:np.float64(-0.005962073802947998) - rollout_corr/ppl_ratio:np.float64(1.0008597834967077) - rollout_corr/chi2_token:np.float64(0.0017287565860897303) - rollout_corr/chi2_seq:np.float64(0.7434897748753428) - actor/pg_loss:np.float64(0.009388480219058692) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001301017089645029) - actor/ppo_kl:np.float64(0.0001421315267471357) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(1917.8817092673853) - self_distillation/token_reweight_w_std:np.float64(43886.33348266815) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036602555774152) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2082537584064994) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.43746187537908554) - perf/mfu/actor:np.float64(0.051655857866616195) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.21465253829956) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003404525574296713 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003404525574296713 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.703125 - response_length/max:972.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:396.703125 - response_length_non_aborted/max:972.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013491325080394745 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8798387572169304) - timing_s/agent_loop/generate_sequences/max:np.float64(8.628671053797007) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.614353348057193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.628671053797007) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:972 - timing_s/gen:15.247638115659356 - timing_s/reward:0.06261933967471123 - timing_s/old_log_prob:3.0381316114217043 - timing_s/adv:0.5441188924014568 - timing_s/update_actor:21.71716057509184 - timing_s/step:40.69797484949231 - timing_s/stop_profile:0.00012857653200626373 - timing_per_token_ms/update_actor:0.12590826149145334 - timing_per_token_ms/gen:0.15014019965003894 - timing_per_token_ms/adv:0.0031546050207639947 - perf/total_num_tokens:172484 - perf/time_per_step:40.69797484949231 - perf/throughput:529.7683749556143 (TaskRunner pid=2255397) Training Progress: 13%|█▎ | 13/100 [12:33<1:26:18, 59.52s/it] (TaskRunner pid=2255397) step:14 - global_seqlen/min:17920 - global_seqlen/max:21796 - global_seqlen/minmax_diff:3876 - global_seqlen/balanced_min:20032 - global_seqlen/balanced_max:20152 - global_seqlen/mean:20054.25 - actor/entropy:0.5020557641983032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18802747130393982 - training/rollout_probs_diff_mean:0.005197192542254925 - training/rollout_probs_diff_std:0.01069364883005619 - training/rollout_actor_probs_pearson_corr:0.9992953538894653 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.40625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.000118374824524 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6423780918121338 - rollout_corr/rollout_is_min:0.6023645997047424 - rollout_corr/rollout_is_std:0.03736548125743866 - rollout_corr/rollout_is_eff_sample_size:0.9986062429080049 - rollout_corr/rollout_is_seq_mean:1.0001424551010132 - rollout_corr/rollout_is_seq_std:0.002069722395390272 - rollout_corr/rollout_is_seq_max:1.0075104236602783 - rollout_corr/rollout_is_seq_min:0.9939577579498291 - rollout_corr/rollout_is_seq_max_deviation:0.00751042366027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7107822382822633) - rollout_corr/training_log_ppl:np.float64(0.5240909514250234) - rollout_corr/kl:np.float64(0.0007750868314850834) - rollout_corr/k3_kl:np.float64(0.0011061159723340097) - rollout_corr/rollout_ppl:np.float64(1.7093983073718846) - rollout_corr/rollout_log_ppl:np.float64(0.5233158656046726) - rollout_corr/log_ppl_diff:np.float64(0.000775085820350796) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019063884974457324) - rollout_corr/log_ppl_diff_max:np.float64(0.009017229080200195) - rollout_corr/log_ppl_diff_min:np.float64(-0.007934510707855225) - rollout_corr/ppl_ratio:np.float64(1.0007783074397594) - rollout_corr/chi2_token:np.float64(0.0027928389608860016) - rollout_corr/chi2_seq:np.float64(0.6327834515832365) - actor/pg_loss:np.float64(0.009141277987509966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0023054060643517005) - actor/ppo_kl:np.float64(0.00017033189165172047) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40625) - self_distillation/token_reweight_w_mean:np.float64(51790.3643477282) - self_distillation/token_reweight_w_std:np.float64(710581.8369147222) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0032089522574097) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1833768331562169) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5500108152627945) - perf/mfu/actor:np.float64(0.04745480566246875) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.28086853027344) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:1.6309666534652933e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:1.6309666534652933e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:359.2578125 - response_length/max:1234.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:359.2578125 - response_length_non_aborted/max:1234.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013267621397972107 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5782812405377626) - timing_s/agent_loop/generate_sequences/max:np.float64(9.458859227597713) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9894911859955755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.458859227597713) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1234 - timing_s/gen:16.56269684806466 - timing_s/reward:0.05258593708276749 - timing_s/old_log_prob:2.994526704773307 - timing_s/adv:0.5849139746278524 - timing_s/update_actor:21.962995065376163 - timing_s/step:42.20455348305404 - timing_s/stop_profile:0.0001186169683933258 - timing_per_token_ms/update_actor:0.13689738500178367 - timing_per_token_ms/gen:0.18008803792611353 - timing_per_token_ms/adv:0.003645823046410689 - perf/total_num_tokens:160434 - perf/time_per_step:42.20455348305404 - perf/throughput:475.1679225335763 (TaskRunner pid=2255397) Training Progress: 14%|█▍ | 14/100 [13:15<1:17:50, 54.31s/it] (TaskRunner pid=2255397) step:15 - global_seqlen/min:17292 - global_seqlen/max:23462 - global_seqlen/minmax_diff:6170 - global_seqlen/balanced_min:20153 - global_seqlen/balanced_max:20223 - global_seqlen/mean:20173.0 - actor/entropy:0.4989277422428131 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23447632789611816 - training/rollout_probs_diff_mean:0.005335042718797922 - training/rollout_probs_diff_std:0.011061806231737137 - training/rollout_actor_probs_pearson_corr:0.99927818775177 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9997453093528748 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5897420644760132 - rollout_corr/rollout_is_min:0.5000474452972412 - rollout_corr/rollout_is_std:0.03860742598772049 - rollout_corr/rollout_is_eff_sample_size:0.9985110313307147 - rollout_corr/rollout_is_seq_mean:0.9997358918190002 - rollout_corr/rollout_is_seq_std:0.0024659933988004923 - rollout_corr/rollout_is_seq_max:1.0084636211395264 - rollout_corr/rollout_is_seq_min:0.9930576682090759 - rollout_corr/rollout_is_seq_max_deviation:0.008463621139526367 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7213433790020645) - rollout_corr/training_log_ppl:np.float64(0.530678162467666) - rollout_corr/kl:np.float64(0.0010814671836847367) - rollout_corr/k3_kl:np.float64(0.000888961919315534) - rollout_corr/rollout_ppl:np.float64(1.7193831540644169) - rollout_corr/rollout_log_ppl:np.float64(0.5295966898556799) - rollout_corr/log_ppl_diff:np.float64(0.0010814726119861007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019996468909084797) - rollout_corr/log_ppl_diff_max:np.float64(0.012681245803833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0078064799308776855) - rollout_corr/ppl_ratio:np.float64(1.0010852627456188) - rollout_corr/chi2_token:np.float64(0.0014137444086372852) - rollout_corr/chi2_seq:np.float64(0.5360429051797837) - actor/pg_loss:np.float64(0.007955617504194379) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006377134768627002) - actor/ppo_kl:np.float64(1.769121835337728e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(8754.255326425191) - self_distillation/token_reweight_w_std:np.float64(117017.50059508975) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001879719318822) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1550164521613624) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.4198375418782234) - perf/mfu/actor:np.float64(0.04879610399473919) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.12841796875) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00847491342574358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00847491342574358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.5 - response_length/max:1036.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.5 - response_length_non_aborted/max:1036.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012453272938728333 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3633983470499516) - timing_s/agent_loop/generate_sequences/max:np.float64(8.056044036522508) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.874184662854532) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.056044036522508) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1036 - timing_s/gen:14.75450462475419 - timing_s/reward:0.052955711260437965 - timing_s/old_log_prob:3.0404567383229733 - timing_s/adv:0.516296835616231 - timing_s/update_actor:21.39274295605719 - timing_s/step:39.795078329741955 - timing_s/stop_profile:3.5120174288749695e-05 - timing_per_token_ms/update_actor:0.13255801663149502 - timing_per_token_ms/gen:0.16778685208281238 - timing_per_token_ms/adv:0.003199182295743264 - perf/total_num_tokens:161384 - perf/time_per_step:39.795078329741955 - perf/throughput:506.9219824835261 (TaskRunner pid=2255397) Training Progress: 15%|█▌ | 15/100 [13:55<1:10:45, 49.94s/it] (TaskRunner pid=2255397) step:16 - global_seqlen/min:17365 - global_seqlen/max:24430 - global_seqlen/minmax_diff:7065 - global_seqlen/balanced_min:20809 - global_seqlen/balanced_max:20815 - global_seqlen/mean:20814.0 - actor/entropy:0.4525703191757202 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44003361463546753 - training/rollout_probs_diff_mean:0.0051152328960597515 - training/rollout_probs_diff_std:0.011277093552052975 - training/rollout_actor_probs_pearson_corr:0.9991934299468994 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:1.000049352645874 - rollout_corr/rollout_is_ratio_fraction_high:1.0565463526290841e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.339278479572386e-05 - rollout_corr/rollout_is_max:2.1190927028656006 - rollout_corr/rollout_is_min:0.16761082410812378 - rollout_corr/rollout_is_std:0.03827483579516411 - rollout_corr/rollout_is_eff_sample_size:0.9985372988785958 - rollout_corr/rollout_is_seq_mean:1.0000265836715698 - rollout_corr/rollout_is_seq_std:0.0022894139401614666 - rollout_corr/rollout_is_seq_max:1.0063250064849854 - rollout_corr/rollout_is_seq_min:0.992612361907959 - rollout_corr/rollout_is_seq_max_deviation:0.007387638092041016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6002246476709843) - rollout_corr/training_log_ppl:np.float64(0.46000494839972816) - rollout_corr/kl:np.float64(0.0006955834787873982) - rollout_corr/k3_kl:np.float64(0.0009704830327450509) - rollout_corr/rollout_ppl:np.float64(1.5991095416247845) - rollout_corr/rollout_log_ppl:np.float64(0.45930936612421647) - rollout_corr/log_ppl_diff:np.float64(0.0006955822755116969) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021134040143806487) - rollout_corr/log_ppl_diff_max:np.float64(0.007766246795654297) - rollout_corr/log_ppl_diff_min:np.float64(-0.005490928888320923) - rollout_corr/ppl_ratio:np.float64(1.0006991014815867) - rollout_corr/chi2_token:np.float64(0.0025813004467636347) - rollout_corr/chi2_seq:np.float64(3.1832106430083513) - actor/pg_loss:np.float64(0.009350149193778634) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011533425140441977) - actor/ppo_kl:np.float64(-8.012323441874969e-05) - actor/pg_clipfrac_lower:np.float64(1.187310044770129e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(35795.58769591758) - self_distillation/token_reweight_w_std:np.float64(482687.72877264104) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030966859776527) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18662144461995922) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.49581966549158096) - perf/mfu/actor:np.float64(0.05050208271241423) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.1108741760254) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01058791484683752 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01058791484683752 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.71875 - response_length/max:1008.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.71875 - response_length_non_aborted/max:1008.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.693439066410065e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3586726989597082) - timing_s/agent_loop/generate_sequences/max:np.float64(7.759539771825075) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9693014876902453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.759539771825075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:1008 - timing_s/gen:14.533444579690695 - timing_s/reward:0.06137754209339619 - timing_s/old_log_prob:3.1735431142151356 - timing_s/adv:0.5521691609174013 - timing_s/update_actor:21.5248114336282 - timing_s/step:39.9321037363261 - timing_s/stop_profile:0.00011518038809299469 - timing_per_token_ms/update_actor:0.1292688300760798 - timing_per_token_ms/gen:0.15355257987163695 - timing_per_token_ms/adv:0.0033160922991580266 - perf/total_num_tokens:166512 - perf/time_per_step:39.9321037363261 - perf/throughput:521.2347472959601 (TaskRunner pid=2255397) Training Progress: 16%|█▌ | 16/100 [14:35<1:05:43, 46.94s/it] (TaskRunner pid=2255397) step:17 - global_seqlen/min:22601 - global_seqlen/max:25800 - global_seqlen/minmax_diff:3199 - global_seqlen/balanced_min:23884 - global_seqlen/balanced_max:24138 - global_seqlen/mean:23919.75 - actor/entropy:0.5084906220436096 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23104846477508545 - training/rollout_probs_diff_mean:0.004854131024330854 - training/rollout_probs_diff_std:0.010198368690907955 - training/rollout_actor_probs_pearson_corr:0.9993425011634827 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8287475288379937e-05 - rollout_corr/rollout_is_max:1.7099158763885498 - rollout_corr/rollout_is_min:0.4197293221950531 - rollout_corr/rollout_is_std:0.036216843873262405 - rollout_corr/rollout_is_eff_sample_size:0.9986904150765131 - rollout_corr/rollout_is_seq_mean:1.0001122951507568 - rollout_corr/rollout_is_seq_std:0.0019326574401929975 - rollout_corr/rollout_is_seq_max:1.0060887336730957 - rollout_corr/rollout_is_seq_min:0.9933558106422424 - rollout_corr/rollout_is_seq_max_deviation:0.006644189357757568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7412785273045301) - rollout_corr/training_log_ppl:np.float64(0.5394562017754652) - rollout_corr/kl:np.float64(0.00046429189457519726) - rollout_corr/k3_kl:np.float64(0.0007774811874128318) - rollout_corr/rollout_ppl:np.float64(1.7404227014631033) - rollout_corr/rollout_log_ppl:np.float64(0.538991907320451) - rollout_corr/log_ppl_diff:np.float64(0.0004642944550141692) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017179182032123208) - rollout_corr/log_ppl_diff_max:np.float64(0.007203280925750732) - rollout_corr/log_ppl_diff_min:np.float64(-0.006280422210693359) - rollout_corr/ppl_ratio:np.float64(1.0004668079782277) - rollout_corr/chi2_token:np.float64(0.002199894981458783) - rollout_corr/chi2_seq:np.float64(2.1964850607328117) - actor/pg_loss:np.float64(0.006313061923719943) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043986083028357825) - actor/ppo_kl:np.float64(-0.00013203885243839864) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(1803.160887064878) - self_distillation/token_reweight_w_std:np.float64(34584.773948686314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014728792011738) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12900599383283406) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.38481806963682175) - perf/mfu/actor:np.float64(0.05734825677685727) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.12204360961914) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005190751980990171 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005190751980990171 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.2734375 - response_length/max:1415.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.2734375 - response_length_non_aborted/max:1415.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010232068598270416 - timing_s/agent_loop/generate_sequences/min:np.float64(1.907282380387187) - timing_s/agent_loop/generate_sequences/max:np.float64(11.15155959688127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.6968326987480395) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.15155959688127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1415 - timing_s/gen:17.835143700242043 - timing_s/reward:0.06522902101278305 - timing_s/old_log_prob:3.041203547269106 - timing_s/adv:0.5475675780326128 - timing_s/update_actor:21.574721600860357 - timing_s/step:43.15138044394553 - timing_s/stop_profile:4.641711711883545e-05 - timing_per_token_ms/update_actor:0.11274533388131333 - timing_per_token_ms/gen:0.16817040093011149 - timing_per_token_ms/adv:0.002861482551200435 - perf/total_num_tokens:191358 - perf/time_per_step:43.15138044394553 - perf/throughput:554.3217796026761 (TaskRunner pid=2255397) Training Progress: 17%|█▋ | 17/100 [15:18<1:03:22, 45.81s/it] (TaskRunner pid=2255397) step:18 - global_seqlen/min:17607 - global_seqlen/max:24855 - global_seqlen/minmax_diff:7248 - global_seqlen/balanced_min:20730 - global_seqlen/balanced_max:20735 - global_seqlen/mean:20733.125 - actor/entropy:0.4945189952850342 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30840808153152466 - training/rollout_probs_diff_mean:0.005118089262396097 - training/rollout_probs_diff_std:0.010652771219611168 - training/rollout_actor_probs_pearson_corr:0.9993004202842712 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9998429417610168 - rollout_corr/rollout_is_ratio_fraction_high:2.0853969544987194e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.032266139984131 - rollout_corr/rollout_is_min:0.5130106210708618 - rollout_corr/rollout_is_std:0.03713827580213547 - rollout_corr/rollout_is_eff_sample_size:0.9986221727100675 - rollout_corr/rollout_is_seq_mean:0.9997930526733398 - rollout_corr/rollout_is_seq_std:0.0022385723423212767 - rollout_corr/rollout_is_seq_max:1.007550835609436 - rollout_corr/rollout_is_seq_min:0.9926072359085083 - rollout_corr/rollout_is_seq_max_deviation:0.007550835609436035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6632476961240172) - rollout_corr/training_log_ppl:np.float64(0.49761353904614225) - rollout_corr/kl:np.float64(0.0008254678116763614) - rollout_corr/k3_kl:np.float64(0.0008354976650934987) - rollout_corr/rollout_ppl:np.float64(1.6618767492473125) - rollout_corr/rollout_log_ppl:np.float64(0.49678807152668014) - rollout_corr/log_ppl_diff:np.float64(0.0008254675194621086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019702696008607745) - rollout_corr/log_ppl_diff_max:np.float64(0.008582055568695068) - rollout_corr/log_ppl_diff_min:np.float64(-0.006475538015365601) - rollout_corr/ppl_ratio:np.float64(1.0008287394884974) - rollout_corr/chi2_token:np.float64(0.0016966825351119041) - rollout_corr/chi2_seq:np.float64(0.7993498002178967) - actor/pg_loss:np.float64(0.006876209634356201) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009630748777453846) - actor/ppo_kl:np.float64(-0.00010461421089758005) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(6797.09068774269) - self_distillation/token_reweight_w_std:np.float64(117938.16381233273) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013849884271622) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17171310502453707) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4298411011695862) - perf/mfu/actor:np.float64(0.05005069497056926) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.3214340209961) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005354256834834814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005354256834834814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.62890625 - response_length/max:907.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.62890625 - response_length_non_aborted/max:907.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001042727380990982 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7595505379140377) - timing_s/agent_loop/generate_sequences/max:np.float64(7.9033639673143625) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.420700289687375) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.9033639673143625) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:907 - timing_s/gen:14.91362101584673 - timing_s/reward:0.05069681629538536 - timing_s/old_log_prob:3.109211662784219 - timing_s/adv:0.5708821713924408 - timing_s/update_actor:21.388794276863337 - timing_s/step:40.0685121640563 - timing_s/stop_profile:0.00011024437844753265 - timing_per_token_ms/update_actor:0.12895302973420153 - timing_per_token_ms/gen:0.15550410318384578 - timing_per_token_ms/adv:0.0034418483187679183 - perf/total_num_tokens:165865 - perf/time_per_step:40.0685121640563 - perf/throughput:517.4418484796842 (TaskRunner pid=2255397) Training Progress: 18%|█▊ | 18/100 [15:58<1:00:16, 44.11s/it] (TaskRunner pid=2255397) step:19 - global_seqlen/min:16539 - global_seqlen/max:26420 - global_seqlen/minmax_diff:9881 - global_seqlen/balanced_min:21542 - global_seqlen/balanced_max:21548 - global_seqlen/mean:21545.625 - actor/entropy:0.48952940106391907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21504220366477966 - training/rollout_probs_diff_mean:0.0051950011402368546 - training/rollout_probs_diff_std:0.010906816460192204 - training/rollout_actor_probs_pearson_corr:0.9992679953575134 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002551078796387 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.993241310119629 - rollout_corr/rollout_is_min:0.508678138256073 - rollout_corr/rollout_is_std:0.03832463547587395 - rollout_corr/rollout_is_eff_sample_size:0.9985340896470543 - rollout_corr/rollout_is_seq_mean:1.000322699546814 - rollout_corr/rollout_is_seq_std:0.0022897543385624886 - rollout_corr/rollout_is_seq_max:1.007610559463501 - rollout_corr/rollout_is_seq_min:0.9937204718589783 - rollout_corr/rollout_is_seq_max_deviation:0.0076105594635009766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6905396389774978) - rollout_corr/training_log_ppl:np.float64(0.5122466792818159) - rollout_corr/kl:np.float64(0.0005300049446929078) - rollout_corr/k3_kl:np.float64(0.0008489201253496503) - rollout_corr/rollout_ppl:np.float64(1.6896098447032273) - rollout_corr/rollout_log_ppl:np.float64(0.511716675653588) - rollout_corr/log_ppl_diff:np.float64(0.0005300036282278597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018085126648657024) - rollout_corr/log_ppl_diff_max:np.float64(0.011858522891998291) - rollout_corr/log_ppl_diff_min:np.float64(-0.007662832736968994) - rollout_corr/ppl_ratio:np.float64(1.0005329437553883) - rollout_corr/chi2_token:np.float64(0.00233305711299181) - rollout_corr/chi2_seq:np.float64(0.9715974144637585) - actor/pg_loss:np.float64(0.006131203263066709) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005582695239354507) - actor/ppo_kl:np.float64(7.291653415109067e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(19979.935181184206) - self_distillation/token_reweight_w_std:np.float64(310836.0626574742) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012645116075873) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1186036104336381) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.3470497876405716) - perf/mfu/actor:np.float64(0.051614476906136436) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.35358047485352) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004195125307887793 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004195125307887793 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:371.17578125 - response_length/max:894.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:371.17578125 - response_length_non_aborted/max:894.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014657527208328247 - timing_s/agent_loop/generate_sequences/min:np.float64(1.919897235929966) - timing_s/agent_loop/generate_sequences/max:np.float64(7.8232190404087305) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320013291013311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.8232190404087305) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:894 - timing_s/gen:14.59238414466381 - timing_s/reward:0.06268099136650562 - timing_s/old_log_prob:2.9667536336928606 - timing_s/adv:0.558667978271842 - timing_s/update_actor:21.68359724804759 - timing_s/step:39.951241597533226 - timing_s/stop_profile:0.00011624395847320557 - timing_per_token_ms/update_actor:0.12580046557043245 - timing_per_token_ms/gen:0.153570096554065 - timing_per_token_ms/adv:0.0032411915311800076 - perf/total_num_tokens:172365 - perf/time_per_step:39.951241597533226 - perf/throughput:539.2980077327641 (TaskRunner pid=2255397) Training Progress: 19%|█▉ | 19/100 [16:38<57:52, 42.87s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we must identify the structure of the product and match it with the most appropriate reactant that would lead to its formation through a plausible synthetic pathway. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key structural features: (TaskRunner pid=2255397) - A **nitrogen-containing ring** (likely a pyrrole or imidazole-like structure). (TaskRunner pid=2255397) - A **chlorinated aromatic ring** (e.g., **Cl-phenyl**). (TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F). (TaskRunner pid=2255397) - A **nitrogen atom** bonded to a carbon chain that includes the **Cl-phenyl** and **trifluoromethyl** groups. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let’s examine the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2255397) This structure contains a **nitrogen ring** with the same **Cl-phenyl** and **trifluoromethyl** substituents as the product. The **carbonyl groups** and **nitrogen ring** suggest a possible **amide** or **nitrile** structure that could be a **precursor** to the product. It is the **most structurally similar** to the product and would likely be the **correct reactant** in a synthesis involving ring formation or substitution. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1` (TaskRunner pid=2255397) This structure contains **two chlorines** and a **nitrogen ring**, but it lacks the **trifluoromethyl group** and the **correct arrangement of substituents** as in the product. It is **not a good match**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` (TaskRunner pid=2255397) This structure includes a **trifluoromethyl group** and **Cl-phenyl**, but the **nitrogen ring** is **not directly bonded to the Cl-phenyl** or **trifluoromethyl**, and the **structure is more complex**. It might be a **byproduct** or **intermediate**, but not the **direct reactant**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F` (TaskRunner pid=2255397) This structure contains a **trifluoromethyl group**, but it **lacks the nitrogen ring** and the **Cl-phenyl** substituents. It is **structurally different** from the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion**: (TaskRunner pid=2255397) Only **Option A** contains the **correct substituents** (Cl-phenyl, trifluoromethyl, and a nitrogen ring), and the **structure is most consistent** with the product. It is the **only plausible reactant** that would lead to the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2255397) step:20 - global_seqlen/min:18853 - global_seqlen/max:23232 - global_seqlen/minmax_diff:4379 - global_seqlen/balanced_min:20703 - global_seqlen/balanced_max:20714 - global_seqlen/mean:20709.5 - actor/entropy:0.4852030277252197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3635062277317047 - training/rollout_probs_diff_mean:0.0052037229761481285 - training/rollout_probs_diff_std:0.010991442948579788 - training/rollout_actor_probs_pearson_corr:0.9992546439170837 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0001426935195923 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1487816411536187e-05 - rollout_corr/rollout_is_max:1.6168099641799927 - rollout_corr/rollout_is_min:0.4058627188205719 - rollout_corr/rollout_is_std:0.03768158331513405 - rollout_corr/rollout_is_eff_sample_size:0.9985827057725277 - rollout_corr/rollout_is_seq_mean:1.0002155303955078 - rollout_corr/rollout_is_seq_std:0.002177149523049593 - rollout_corr/rollout_is_seq_max:1.0087448358535767 - rollout_corr/rollout_is_seq_min:0.9949045181274414 - rollout_corr/rollout_is_seq_max_deviation:0.00874483585357666 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.65907796099782) - rollout_corr/training_log_ppl:np.float64(0.49622454700875096) - rollout_corr/kl:np.float64(0.0007081711968908166) - rollout_corr/k3_kl:np.float64(0.000836590931498904) - rollout_corr/rollout_ppl:np.float64(1.6578862462192774) - rollout_corr/rollout_log_ppl:np.float64(0.49551637563854456) - rollout_corr/log_ppl_diff:np.float64(0.0007081713702064008) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018356915388721973) - rollout_corr/log_ppl_diff_max:np.float64(0.007012069225311279) - rollout_corr/log_ppl_diff_min:np.float64(-0.006520330905914307) - rollout_corr/ppl_ratio:np.float64(1.0007108724676073) - rollout_corr/chi2_token:np.float64(0.001943168230354786) - rollout_corr/chi2_seq:np.float64(0.7969548560213298) - actor/pg_loss:np.float64(0.006117190117947757) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006000254938953731) - actor/ppo_kl:np.float64(0.0001683622528645401) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(23049.971880125348) - self_distillation/token_reweight_w_std:np.float64(317665.6890155331) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982667530421168) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15381890590651892) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3626445010304451) - perf/mfu/actor:np.float64(0.05021794625693002) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.33641052246094) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5196428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.2570318204515784) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6278380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21727863221866373) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41209523809523807) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2203367172556475) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5193428571428571) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2582634523002606) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.713442857142857) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15828990121163083) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.3252571428571428) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15921732171540784) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5385380952380953) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2095760136213612) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7737142857142857) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1018065135469096) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2657142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10472175061646756) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5479142857142857) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15900279342137813) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8120428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05885508241318426) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.22566190476190473) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06395190241306174) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5533238095238096) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11781703809110204) - val-aux/sciknoweval/score/mean@16:np.float64(0.5196428571428572) - val-aux/sciknoweval/score/std@16:np.float64(0.2570318204515784) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6278380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21727863221866373) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.41209523809523807) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2203367172556475) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5193428571428571) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2582634523002606) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.713442857142857) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15828990121163083) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.3252571428571428) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15921732171540784) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5385380952380953) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2095760136213612) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7737142857142857) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1018065135469096) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.2657142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10472175061646756) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5479142857142857) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15900279342137813) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8120428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05885508241318426) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.22566190476190473) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06395190241306174) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5533238095238096) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11781703809110204) - val-core/sciknoweval/acc/mean@16:np.float64(0.5196428571428572) - val-aux/sciknoweval/acc/std@16:np.float64(0.2570318204515784) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6278380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21727863221866373) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.41209523809523807) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2203367172556475) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5193428571428571) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2582634523002606) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.713442857142857) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15828990121163083) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.3252571428571428) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15921732171540784) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5385380952380953) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2095760136213612) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7737142857142857) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1018065135469096) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.2657142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10472175061646756) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5479142857142857) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15900279342137813) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8120428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05885508241318426) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.22566190476190473) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06395190241306174) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5533238095238096) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11781703809110204) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9979166666666667) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.005738829557423338) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9996761904761906) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0021556297891449894) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9960809523809524) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.007408678245820718) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9977904761904762) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0058674039295688075) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9931095238095238) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008738507306529836) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9991428571428571) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0038031567601600064) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9892238095238096) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.008745323711299243) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997809523809524) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0019148844274678545) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9851285714285715) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006912686317569252) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00036325512888540755) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004073284100741148 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004073284100741148 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:363.578125 - response_length/max:1073.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:363.578125 - response_length_non_aborted/max:1073.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001299697905778885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8756434936076403) - timing_s/agent_loop/generate_sequences/max:np.float64(8.891039799898863) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.303868516799412) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.891039799898863) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1073 - timing_s/gen:15.654097167775035 - timing_s/reward:0.05749098025262356 - timing_s/old_log_prob:3.016568712890148 - timing_s/adv:0.5642522405833006 - timing_s/update_actor:21.53763866983354 - timing_s/step:40.86700990051031 - timing_s/testing:131.59580136090517 - timing_s/save_checkpoint:13.167504344135523 - timing_s/stop_profile:0.00014436431229114532 - timing_per_token_ms/update_actor:0.12999854336073746 - timing_per_token_ms/gen:0.16818618298782753 - timing_per_token_ms/adv:0.003405757264681068 - perf/total_num_tokens:165676 - perf/time_per_step:40.86700990051031 - perf/throughput:506.7534926195175 (TaskRunner pid=2255397) Training Progress: 20%|██ | 20/100 [19:44<1:54:20, 85.76s/it] (TaskRunner pid=2255397) step:21 - global_seqlen/min:17376 - global_seqlen/max:26208 - global_seqlen/minmax_diff:8832 - global_seqlen/balanced_min:22075 - global_seqlen/balanced_max:22088 - global_seqlen/mean:22085.25 - actor/entropy:0.4656538665294647 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102373421192169 - training/rollout_probs_diff_mean:0.004885641857981682 - training/rollout_probs_diff_std:0.010877853259444237 - training/rollout_actor_probs_pearson_corr:0.9992596507072449 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.84765625 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84765625 - rollout_corr/rollout_is_mean:0.9998842477798462 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.879465500882361e-05 - rollout_corr/rollout_is_max:1.9200012683868408 - rollout_corr/rollout_is_min:0.25445759296417236 - rollout_corr/rollout_is_std:0.03666176274418831 - rollout_corr/rollout_is_eff_sample_size:0.9986574816202841 - rollout_corr/rollout_is_seq_mean:0.9998617172241211 - rollout_corr/rollout_is_seq_std:0.0022321075666695833 - rollout_corr/rollout_is_seq_max:1.0074141025543213 - rollout_corr/rollout_is_seq_min:0.9920339584350586 - rollout_corr/rollout_is_seq_max_deviation:0.007966041564941406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6367876464501023) - rollout_corr/training_log_ppl:np.float64(0.47835954203037545) - rollout_corr/kl:np.float64(0.0009155909243130367) - rollout_corr/k3_kl:np.float64(0.0008391680780164279) - rollout_corr/rollout_ppl:np.float64(1.6352251395583153) - rollout_corr/rollout_log_ppl:np.float64(0.4774439476314001) - rollout_corr/log_ppl_diff:np.float64(0.0009155943989753723) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018019898561760783) - rollout_corr/log_ppl_diff_max:np.float64(0.008913397789001465) - rollout_corr/log_ppl_diff_min:np.float64(-0.006456807255744934) - rollout_corr/ppl_ratio:np.float64(1.0009184170048684) - rollout_corr/chi2_token:np.float64(0.001522636041045189) - rollout_corr/chi2_seq:np.float64(0.3302574837580323) - actor/pg_loss:np.float64(0.00977602880448103) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008731294096833153) - actor/ppo_kl:np.float64(2.3401729779659597e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(19834.05608875933) - self_distillation/token_reweight_w_std:np.float64(292799.855937861) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0034907914232463) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1998413290712051) - self_distillation/empty_target_batch:np.float64(0.15234375) - actor/grad_norm:np.float64(0.506427951157093) - perf/mfu/actor:np.float64(0.05293222627939937) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.2050437927246) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00826286617666483 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00826286617666483 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.9765625 - response_length/max:1132.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.9765625 - response_length_non_aborted/max:1132.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.725825369358063e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8232946544885635) - timing_s/agent_loop/generate_sequences/max:np.float64(9.148927487432957) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.626729334348056) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.148927487432957) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:234 - timing_s/agent_loop/slowest/response_length:1132 - timing_s/gen:15.357284164056182 - timing_s/reward:0.06359587237238884 - timing_s/old_log_prob:3.120852567255497 - timing_s/adv:0.6262220405042171 - timing_s/update_actor:21.799812253564596 - timing_s/step:41.07217570580542 - timing_s/stop_profile:7.509812712669373e-05 - timing_per_token_ms/update_actor:0.12338445485994383 - timing_per_token_ms/gen:0.14740257005793658 - timing_per_token_ms/adv:0.0035443454370236763 - perf/total_num_tokens:176682 - perf/time_per_step:41.07217570580542 - perf/throughput:537.7180444053837 (TaskRunner pid=2255397) Training Progress: 21%|██ | 21/100 [20:25<1:35:16, 72.35s/it] (TaskRunner pid=2255397) step:22 - global_seqlen/min:15780 - global_seqlen/max:20714 - global_seqlen/minmax_diff:4934 - global_seqlen/balanced_min:18793 - global_seqlen/balanced_max:18799 - global_seqlen/mean:18797.25 - actor/entropy:0.4407426416873932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23966830968856812 - training/rollout_probs_diff_mean:0.0051012844778597355 - training/rollout_probs_diff_std:0.011158647015690804 - training/rollout_actor_probs_pearson_corr:0.9991984963417053 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.4375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9999550580978394 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.586863398551941 - rollout_corr/rollout_is_min:0.6212320327758789 - rollout_corr/rollout_is_std:0.03741012141108513 - rollout_corr/rollout_is_eff_sample_size:0.9986023199750391 - rollout_corr/rollout_is_seq_mean:0.9999154806137085 - rollout_corr/rollout_is_seq_std:0.0023052047472447157 - rollout_corr/rollout_is_seq_max:1.0075262784957886 - rollout_corr/rollout_is_seq_min:0.9921414256095886 - rollout_corr/rollout_is_seq_max_deviation:0.007858574390411377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6121524260379374) - rollout_corr/training_log_ppl:np.float64(0.46336599756614305) - rollout_corr/kl:np.float64(0.0012106716192334943) - rollout_corr/k3_kl:np.float64(0.001044533465915265) - rollout_corr/rollout_ppl:np.float64(1.6101201642304659) - rollout_corr/rollout_log_ppl:np.float64(0.4621553277829662) - rollout_corr/log_ppl_diff:np.float64(0.0012106697831768543) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022202231630217284) - rollout_corr/log_ppl_diff_max:np.float64(0.011315524578094482) - rollout_corr/log_ppl_diff_min:np.float64(-0.007631242275238037) - rollout_corr/ppl_ratio:np.float64(1.0012149058748037) - rollout_corr/chi2_token:np.float64(0.001704103546217084) - rollout_corr/chi2_seq:np.float64(0.4757801443338394) - actor/pg_loss:np.float64(0.006979008438065648) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011709280256582133) - actor/ppo_kl:np.float64(0.00036738665134805615) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4375) - self_distillation/token_reweight_w_mean:np.float64(53386.58389568445) - self_distillation/token_reweight_w_std:np.float64(762098.1771182105) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011509482283145) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1306488252885174) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.4595351442694664) - perf/mfu/actor:np.float64(0.0457439323763552) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.23744583129883) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024807543959468603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024807543959468603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.8515625 - response_length/max:891.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.8515625 - response_length_non_aborted/max:891.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.7911500334739685e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6936665810644627) - timing_s/agent_loop/generate_sequences/max:np.float64(7.335243610665202) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.780559276121494) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.335243610665202) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:891 - timing_s/gen:14.717649631202221 - timing_s/reward:0.05497334711253643 - timing_s/old_log_prob:2.9507923033088446 - timing_s/adv:0.4809166081249714 - timing_s/update_actor:21.40921910852194 - timing_s/step:39.70510539226234 - timing_s/stop_profile:0.00011640414595603943 - timing_per_token_ms/update_actor:0.1423693566114853 - timing_per_token_ms/gen:0.1732425739953648 - timing_per_token_ms/adv:0.003198051630723719 - perf/total_num_tokens:150378 - perf/time_per_step:39.70510539226234 - perf/throughput:473.42148608584665 (TaskRunner pid=2255397) Training Progress: 22%|██▏ | 22/100 [21:05<1:21:20, 62.57s/it] (TaskRunner pid=2255397) step:23 - global_seqlen/min:15655 - global_seqlen/max:22837 - global_seqlen/minmax_diff:7182 - global_seqlen/balanced_min:19794 - global_seqlen/balanced_max:19804 - global_seqlen/mean:19802.0 - actor/entropy:0.43892350792884827 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22363799810409546 - training/rollout_probs_diff_mean:0.005129741504788399 - training/rollout_probs_diff_std:0.011249431408941746 - training/rollout_actor_probs_pearson_corr:0.9991777539253235 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999939203262329 - rollout_corr/rollout_is_ratio_fraction_high:1.1452129911049269e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1670188903808594 - rollout_corr/rollout_is_min:0.5688050389289856 - rollout_corr/rollout_is_std:0.03843645006418228 - rollout_corr/rollout_is_eff_sample_size:0.9985248186495871 - rollout_corr/rollout_is_seq_mean:0.9999755024909973 - rollout_corr/rollout_is_seq_std:0.0022403132170438766 - rollout_corr/rollout_is_seq_max:1.008310079574585 - rollout_corr/rollout_is_seq_min:0.9935483336448669 - rollout_corr/rollout_is_seq_max_deviation:0.008310079574584961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.585592514835298) - rollout_corr/training_log_ppl:np.float64(0.45206161157693714) - rollout_corr/kl:np.float64(0.0010271025659691624) - rollout_corr/k3_kl:np.float64(0.0009754648472721783) - rollout_corr/rollout_ppl:np.float64(1.583907154854387) - rollout_corr/rollout_log_ppl:np.float64(0.45103450742317364) - rollout_corr/log_ppl_diff:np.float64(0.0010271041537635028) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021158340969122946) - rollout_corr/log_ppl_diff_max:np.float64(0.008697479963302612) - rollout_corr/log_ppl_diff_min:np.float64(-0.007354050874710083) - rollout_corr/ppl_ratio:np.float64(1.0010308369528502) - rollout_corr/chi2_token:np.float64(0.0018195987213402987) - rollout_corr/chi2_seq:np.float64(1.1621388338971883) - actor/pg_loss:np.float64(0.007446382311172783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000959160078309651) - actor/ppo_kl:np.float64(0.00021888582077522756) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(6609.447738413466) - self_distillation/token_reweight_w_std:np.float64(122546.46291852725) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9963779090903699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17003195008146577) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.43443264812231064) - perf/mfu/actor:np.float64(0.04804480134893495) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.1064910888672) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033225491642951965 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033225491642951965 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.09375 - response_length/max:846.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.09375 - response_length_non_aborted/max:846.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010677054524421692 - timing_s/agent_loop/generate_sequences/min:np.float64(1.892102275043726) - timing_s/agent_loop/generate_sequences/max:np.float64(7.085802031680942) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9653804742338252) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.085802031680942) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:224 - timing_s/agent_loop/slowest/response_length:846 - timing_s/gen:13.813347652554512 - timing_s/reward:0.060129137709736824 - timing_s/old_log_prob:2.9367834143340588 - timing_s/adv:0.5313698258250952 - timing_s/update_actor:21.404147198423743 - timing_s/step:38.83369122073054 - timing_s/stop_profile:0.00010784715414047241 - timing_per_token_ms/update_actor:0.1351135440765058 - timing_per_token_ms/gen:0.15819225438106405 - timing_per_token_ms/adv:0.0033542686712522423 - perf/total_num_tokens:158416 - perf/time_per_step:38.83369122073054 - perf/throughput:509.91804738430636 (TaskRunner pid=2255397) Training Progress: 23%|██▎ | 23/100 [21:44<1:11:10, 55.46s/it] (TaskRunner pid=2255397) step:24 - global_seqlen/min:18708 - global_seqlen/max:22807 - global_seqlen/minmax_diff:4099 - global_seqlen/balanced_min:20352 - global_seqlen/balanced_max:20355 - global_seqlen/mean:20353.75 - actor/entropy:0.48541581630706787 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16695591807365417 - training/rollout_probs_diff_mean:0.005245619919151068 - training/rollout_probs_diff_std:0.01105139683932066 - training/rollout_actor_probs_pearson_corr:0.9992536306381226 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9997487664222717 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6379640102386475 - rollout_corr/rollout_is_min:0.55048668384552 - rollout_corr/rollout_is_std:0.038326967507600784 - rollout_corr/rollout_is_eff_sample_size:0.998532425609191 - rollout_corr/rollout_is_seq_mean:0.9996849894523621 - rollout_corr/rollout_is_seq_std:0.0021940250881016254 - rollout_corr/rollout_is_seq_max:1.0061367750167847 - rollout_corr/rollout_is_seq_min:0.9898834228515625 - rollout_corr/rollout_is_seq_max_deviation:0.0101165771484375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6475044381804764) - rollout_corr/training_log_ppl:np.float64(0.4905120711773634) - rollout_corr/kl:np.float64(0.0011423102059779922) - rollout_corr/k3_kl:np.float64(0.0009413222521743592) - rollout_corr/rollout_ppl:np.float64(1.6456106547266245) - rollout_corr/rollout_log_ppl:np.float64(0.48936975869582966) - rollout_corr/log_ppl_diff:np.float64(0.001142312481533736) - rollout_corr/log_ppl_abs_diff:np.float64(0.002205758646596223) - rollout_corr/log_ppl_diff_max:np.float64(0.011655747890472412) - rollout_corr/log_ppl_diff_min:np.float64(-0.006167978048324585) - rollout_corr/ppl_ratio:np.float64(1.0011465374846011) - rollout_corr/chi2_token:np.float64(0.001514944015070796) - rollout_corr/chi2_seq:np.float64(0.9692259337753057) - actor/pg_loss:np.float64(0.006828168756328523) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008421741476922762) - actor/ppo_kl:np.float64(5.722692924337025e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(16169.235121513018) - self_distillation/token_reweight_w_std:np.float64(204586.64157430374) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000086387153715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14809982472797856) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.38066764548420906) - perf/mfu/actor:np.float64(0.04935736118807353) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.85000133514404) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-8.236870780820027e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-8.236870780820027e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.6796875 - response_length/max:641.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.6796875 - response_length_non_aborted/max:641.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011152960360050201 - timing_s/agent_loop/generate_sequences/min:np.float64(2.046153662726283) - timing_s/agent_loop/generate_sequences/max:np.float64(6.436841234564781) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.186256590895937) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.436841234564781) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:234 - timing_s/agent_loop/slowest/response_length:641 - timing_s/gen:13.05754235945642 - timing_s/reward:0.05888623557984829 - timing_s/old_log_prob:2.9728885516524315 - timing_s/adv:0.5305459387600422 - timing_s/update_actor:21.35632175952196 - timing_s/step:38.063012946397066 - timing_s/stop_profile:0.00010835938155651093 - timing_per_token_ms/update_actor:0.13115716857779255 - timing_per_token_ms/gen:0.14340437937329958 - timing_per_token_ms/adv:0.0032582812673342884 - perf/total_num_tokens:162830 - perf/time_per_step:38.063012946397066 - perf/throughput:534.7382780407726 (TaskRunner pid=2255397) Training Progress: 24%|██▍ | 24/100 [22:22<1:03:39, 50.25s/it] (TaskRunner pid=2255397) step:25 - global_seqlen/min:16381 - global_seqlen/max:25188 - global_seqlen/minmax_diff:8807 - global_seqlen/balanced_min:21088 - global_seqlen/balanced_max:21097 - global_seqlen/mean:21093.125 - actor/entropy:0.4773712158203125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26223766803741455 - training/rollout_probs_diff_mean:0.005165540147572756 - training/rollout_probs_diff_std:0.01117149367928505 - training/rollout_actor_probs_pearson_corr:0.9992371797561646 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999747276306152 - rollout_corr/rollout_is_ratio_fraction_high:1.0959985047520604e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.094183921813965 - rollout_corr/rollout_is_min:0.5062851309776306 - rollout_corr/rollout_is_std:0.038087502121925354 - rollout_corr/rollout_is_eff_sample_size:0.9985513246511216 - rollout_corr/rollout_is_seq_mean:1.0000536441802979 - rollout_corr/rollout_is_seq_std:0.0022864702623337507 - rollout_corr/rollout_is_seq_max:1.0067065954208374 - rollout_corr/rollout_is_seq_min:0.9937045574188232 - rollout_corr/rollout_is_seq_max_deviation:0.006706595420837402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.669403042178601) - rollout_corr/training_log_ppl:np.float64(0.4993552331288811) - rollout_corr/kl:np.float64(0.0006479631488460758) - rollout_corr/k3_kl:np.float64(0.0008453664596572708) - rollout_corr/rollout_ppl:np.float64(1.668324178084731) - rollout_corr/rollout_log_ppl:np.float64(0.4987072692310903) - rollout_corr/log_ppl_diff:np.float64(0.0006479638977907598) - rollout_corr/log_ppl_abs_diff:np.float64(0.001830984663683921) - rollout_corr/log_ppl_diff_max:np.float64(0.008455544710159302) - rollout_corr/log_ppl_diff_min:np.float64(-0.007198035717010498) - rollout_corr/ppl_ratio:np.float64(1.0006508629303426) - rollout_corr/chi2_token:np.float64(0.002105830702930689) - rollout_corr/chi2_seq:np.float64(0.6466955123469234) - actor/pg_loss:np.float64(0.007964683580212295) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000689070585394802) - actor/ppo_kl:np.float64(-6.814886182837654e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(26990.30881792493) - self_distillation/token_reweight_w_std:np.float64(421488.5579388592) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002008835086599) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15447656423202716) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4783755913376808) - perf/mfu/actor:np.float64(0.0494590320538676) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.90256881713867) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02120209112763405 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02120209112763405 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:356.41015625 - response_length/max:956.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:356.41015625 - response_length_non_aborted/max:956.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000149555504322052 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6570597048848867) - timing_s/agent_loop/generate_sequences/max:np.float64(7.838871484622359) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9465300647134427) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.838871484622359) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:956 - timing_s/gen:14.78430269099772 - timing_s/reward:0.06230742111802101 - timing_s/old_log_prob:3.0198799911886454 - timing_s/adv:0.5167359504848719 - timing_s/update_actor:22.208827931433916 - timing_s/step:40.68130003660917 - timing_s/stop_profile:0.0001026783138513565 - timing_per_token_ms/update_actor:0.13161176883127745 - timing_per_token_ms/gen:0.16203573712473252 - timing_per_token_ms/adv:0.003062229698568087 - perf/total_num_tokens:168745 - perf/time_per_step:40.68130003660917 - perf/throughput:518.496827314227 (TaskRunner pid=2255397) Training Progress: 25%|██▌ | 25/100 [23:03<59:14, 47.40s/it] (TaskRunner pid=2255397) step:26 - global_seqlen/min:18171 - global_seqlen/max:23062 - global_seqlen/minmax_diff:4891 - global_seqlen/balanced_min:20385 - global_seqlen/balanced_max:20394 - global_seqlen/mean:20391.25 - actor/entropy:0.5068786144256592 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17874199151992798 - training/rollout_probs_diff_mean:0.005472649820148945 - training/rollout_probs_diff_std:0.011403916403651237 - training/rollout_actor_probs_pearson_corr:0.9991965889930725 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000735521316528 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.292263525305316e-05 - rollout_corr/rollout_is_max:1.8139365911483765 - rollout_corr/rollout_is_min:0.43659546971321106 - rollout_corr/rollout_is_std:0.039478376507759094 - rollout_corr/rollout_is_eff_sample_size:0.9984440018610561 - rollout_corr/rollout_is_seq_mean:1.0000500679016113 - rollout_corr/rollout_is_seq_std:0.0023560679983347654 - rollout_corr/rollout_is_seq_max:1.0076462030410767 - rollout_corr/rollout_is_seq_min:0.9908225536346436 - rollout_corr/rollout_is_seq_max_deviation:0.009177446365356445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6881171250715852) - rollout_corr/training_log_ppl:np.float64(0.510973722091876) - rollout_corr/kl:np.float64(0.0007455622756609159) - rollout_corr/k3_kl:np.float64(0.0008864333188967066) - rollout_corr/rollout_ppl:np.float64(1.6867859680205584) - rollout_corr/rollout_log_ppl:np.float64(0.5102281617000699) - rollout_corr/log_ppl_diff:np.float64(0.000745560391806066) - rollout_corr/log_ppl_abs_diff:np.float64(0.002050930866971612) - rollout_corr/log_ppl_diff_max:np.float64(0.00910806655883789) - rollout_corr/log_ppl_diff_min:np.float64(-0.006127357482910156) - rollout_corr/ppl_ratio:np.float64(1.0007490692660213) - rollout_corr/chi2_token:np.float64(0.0020302317570894957) - rollout_corr/chi2_seq:np.float64(1.0315312510356307) - actor/pg_loss:np.float64(0.005001060431823134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048566531631877297) - actor/ppo_kl:np.float64(-2.1595971896104516e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(13624.06982974615) - self_distillation/token_reweight_w_std:np.float64(206433.80913565995) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992130598984659) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09108548518270254) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3169581815600395) - perf/mfu/actor:np.float64(0.04962560549786612) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.88824081420898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027206302620470524 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027206302620470524 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.8203125 - response_length/max:592.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.8203125 - response_length_non_aborted/max:592.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016283243894577026 - timing_s/agent_loop/generate_sequences/min:np.float64(2.038805991411209) - timing_s/agent_loop/generate_sequences/max:np.float64(5.906553823500872) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0381240394635824) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.906553823500872) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:331 - timing_s/agent_loop/slowest/response_length:592 - timing_s/gen:12.749464539811015 - timing_s/reward:0.061460068449378014 - timing_s/old_log_prob:3.0010161008685827 - timing_s/adv:0.5166078116744757 - timing_s/update_actor:21.353253232315183 - timing_s/step:37.770273169502616 - timing_s/stop_profile:0.00010509975254535675 - timing_per_token_ms/update_actor:0.13089715706684965 - timing_per_token_ms/gen:0.1461256680780632 - timing_per_token_ms/adv:0.0031668473712650994 - perf/total_num_tokens:163130 - perf/time_per_step:37.770273169502616 - perf/throughput:539.8756294001282 (TaskRunner pid=2255397) Training Progress: 26%|██▌ | 26/100 [23:40<54:55, 44.53s/it] (TaskRunner pid=2255397) step:27 - global_seqlen/min:17480 - global_seqlen/max:23456 - global_seqlen/minmax_diff:5976 - global_seqlen/balanced_min:20067 - global_seqlen/balanced_max:20076 - global_seqlen/mean:20073.875 - actor/entropy:0.4675213694572449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.005088143516331911 - training/rollout_probs_diff_std:0.010994603857398033 - training/rollout_actor_probs_pearson_corr:0.9992254972457886 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9997977018356323 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6167669296264648 - rollout_corr/rollout_is_min:0.5378821492195129 - rollout_corr/rollout_is_std:0.03790003061294556 - rollout_corr/rollout_is_eff_sample_size:0.9985651130827546 - rollout_corr/rollout_is_seq_mean:0.9997107982635498 - rollout_corr/rollout_is_seq_std:0.002149628708139062 - rollout_corr/rollout_is_seq_max:1.007951021194458 - rollout_corr/rollout_is_seq_min:0.9923518896102905 - rollout_corr/rollout_is_seq_max_deviation:0.007951021194458008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6248419084586203) - rollout_corr/training_log_ppl:np.float64(0.47598731669131666) - rollout_corr/kl:np.float64(0.0010493483772009427) - rollout_corr/k3_kl:np.float64(0.0008248598063573809) - rollout_corr/rollout_ppl:np.float64(1.623152432963252) - rollout_corr/rollout_log_ppl:np.float64(0.47493796655908227) - rollout_corr/log_ppl_diff:np.float64(0.0010493501322343946) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018501168815419078) - rollout_corr/log_ppl_diff_max:np.float64(0.007712453603744507) - rollout_corr/log_ppl_diff_min:np.float64(-0.004121840000152588) - rollout_corr/ppl_ratio:np.float64(1.0010521123185754) - rollout_corr/chi2_token:np.float64(0.0011714110150933266) - rollout_corr/chi2_seq:np.float64(0.32961429841816425) - actor/pg_loss:np.float64(0.008322581415995955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008014981349333539) - actor/ppo_kl:np.float64(4.5028834745330926e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(30577.34729726822) - self_distillation/token_reweight_w_std:np.float64(521484.4462188269) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020933467894793) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16031731336261146) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.42759493738412857) - perf/mfu/actor:np.float64(0.048367516590069666) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.80517578125) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014940870460122824 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014940870460122824 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.62109375 - response_length/max:943.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.62109375 - response_length_non_aborted/max:943.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013620592653751373 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9484890308231115) - timing_s/agent_loop/generate_sequences/max:np.float64(7.838706968352199) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320610681475955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.838706968352199) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:943 - timing_s/gen:14.69153231009841 - timing_s/reward:0.06106424517929554 - timing_s/old_log_prob:3.0161088667809963 - timing_s/adv:0.5223662443459034 - timing_s/update_actor:21.658704286441207 - timing_s/step:40.0367049369961 - timing_s/stop_profile:0.00010894984006881714 - timing_per_token_ms/update_actor:0.13486873041727873 - timing_per_token_ms/gen:0.15526386090166672 - timing_per_token_ms/adv:0.003252774092856408 - perf/total_num_tokens:160591 - perf/time_per_step:40.0367049369961 - perf/throughput:501.38679073588406 (TaskRunner pid=2255397) Training Progress: 27%|██▋ | 27/100 [24:21<52:33, 43.20s/it] (TaskRunner pid=2255397) step:28 - global_seqlen/min:18201 - global_seqlen/max:23560 - global_seqlen/minmax_diff:5359 - global_seqlen/balanced_min:20410 - global_seqlen/balanced_max:20497 - global_seqlen/mean:20425.125 - actor/entropy:0.4737909138202667 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294958233833313 - training/rollout_probs_diff_mean:0.0052576144225895405 - training/rollout_probs_diff_std:0.011254720389842987 - training/rollout_actor_probs_pearson_corr:0.999194860458374 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001050233840942 - rollout_corr/rollout_is_ratio_fraction_high:1.0676346391846891e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.8465442657470703 - rollout_corr/rollout_is_min:0.6126700639724731 - rollout_corr/rollout_is_std:0.03862826153635979 - rollout_corr/rollout_is_eff_sample_size:0.9985103182038877 - rollout_corr/rollout_is_seq_mean:1.0001041889190674 - rollout_corr/rollout_is_seq_std:0.002421688986942172 - rollout_corr/rollout_is_seq_max:1.00888991355896 - rollout_corr/rollout_is_seq_min:0.9913090467453003 - rollout_corr/rollout_is_seq_max_deviation:0.008889913558959961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6370238508097827) - rollout_corr/training_log_ppl:np.float64(0.4820473712461535) - rollout_corr/kl:np.float64(0.0006596640439546775) - rollout_corr/k3_kl:np.float64(0.000909588395757055) - rollout_corr/rollout_ppl:np.float64(1.6358301592990756) - rollout_corr/rollout_log_ppl:np.float64(0.48138770982041024) - rollout_corr/log_ppl_diff:np.float64(0.000659661425743252) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018744217813946307) - rollout_corr/log_ppl_diff_max:np.float64(0.007977485656738281) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074624717235565186) - rollout_corr/ppl_ratio:np.float64(1.0006625556852669) - rollout_corr/chi2_token:np.float64(0.0023598361294716597) - rollout_corr/chi2_seq:np.float64(0.9705478034447879) - actor/pg_loss:np.float64(0.006433643866330385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006121892383816885) - actor/ppo_kl:np.float64(-2.0443672603676077e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(22090.04282637965) - self_distillation/token_reweight_w_std:np.float64(363460.48439942) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007038170006126) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12272839152137749) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.4047693833708763) - perf/mfu/actor:np.float64(0.0494901405269842) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.81372833251953) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004289222415536642 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004289222415536642 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:365.87890625 - response_length/max:1022.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:365.87890625 - response_length_non_aborted/max:1022.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001571252942085266 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0796666648238897) - timing_s/agent_loop/generate_sequences/max:np.float64(8.435203347355127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.275154207913147) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.435203347355127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:1022 - timing_s/gen:15.208884691819549 - timing_s/reward:0.060688696801662445 - timing_s/old_log_prob:3.0850410480052233 - timing_s/adv:0.5393318682909012 - timing_s/update_actor:21.212657935917377 - timing_s/step:40.19403674826026 - timing_s/stop_profile:0.00011958740651607513 - timing_per_token_ms/update_actor:0.1298196335145891 - timing_per_token_ms/gen:0.1623753236728719 - timing_per_token_ms/adv:0.003300664428558584 - perf/total_num_tokens:163401 - perf/time_per_step:40.19403674826026 - perf/throughput:508.1630672710193 (TaskRunner pid=2255397) Training Progress: 28%|██▊ | 28/100 [25:01<50:46, 42.31s/it] (TaskRunner pid=2255397) step:29 - global_seqlen/min:17728 - global_seqlen/max:24689 - global_seqlen/minmax_diff:6961 - global_seqlen/balanced_min:21875 - global_seqlen/balanced_max:21882 - global_seqlen/mean:21878.875 - actor/entropy:0.45051109790802 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25539880990982056 - training/rollout_probs_diff_mean:0.004729867912828922 - training/rollout_probs_diff_std:0.010498874820768833 - training/rollout_actor_probs_pearson_corr:0.9993234872817993 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9999346733093262 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9196990251657553e-05 - rollout_corr/rollout_is_max:1.6618187427520752 - rollout_corr/rollout_is_min:0.48319488763809204 - rollout_corr/rollout_is_std:0.03575139120221138 - rollout_corr/rollout_is_eff_sample_size:0.998723350704955 - rollout_corr/rollout_is_seq_mean:0.9999228119850159 - rollout_corr/rollout_is_seq_std:0.00180347659625113 - rollout_corr/rollout_is_seq_max:1.0053354501724243 - rollout_corr/rollout_is_seq_min:0.9929618835449219 - rollout_corr/rollout_is_seq_max_deviation:0.007038116455078125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5977162146009505) - rollout_corr/training_log_ppl:np.float64(0.4593473710701801) - rollout_corr/kl:np.float64(0.0006978295002575408) - rollout_corr/k3_kl:np.float64(0.0007481996697720206) - rollout_corr/rollout_ppl:np.float64(1.5965832527726889) - rollout_corr/rollout_log_ppl:np.float64(0.45864954427815974) - rollout_corr/log_ppl_diff:np.float64(0.0006978267920203507) - rollout_corr/log_ppl_abs_diff:np.float64(0.001622283656615764) - rollout_corr/log_ppl_diff_max:np.float64(0.006626904010772705) - rollout_corr/log_ppl_diff_min:np.float64(-0.00486445426940918) - rollout_corr/ppl_ratio:np.float64(1.0007000237237662) - rollout_corr/chi2_token:np.float64(0.001576458802446723) - rollout_corr/chi2_seq:np.float64(0.6520844814367592) - actor/pg_loss:np.float64(0.0060147366020828485) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005360728382584057) - actor/ppo_kl:np.float64(-6.641209190716424e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(26070.862267469056) - self_distillation/token_reweight_w_std:np.float64(478879.7805042105) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025979080237448) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1094071113911923) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.3129398711025715) - perf/mfu/actor:np.float64(0.052564897191348076) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.9274024963379) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01068192534148693 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01068192534148693 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.96484375 - response_length/max:856.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.96484375 - response_length_non_aborted/max:856.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011670216917991638 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0912411771714687) - timing_s/agent_loop/generate_sequences/max:np.float64(7.972104227170348) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.765111505068489) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.972104227170348) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:856 - timing_s/gen:14.771529672667384 - timing_s/reward:0.06256661377847195 - timing_s/old_log_prob:3.17423870973289 - timing_s/adv:0.5342923663556576 - timing_s/update_actor:21.539450408890843 - timing_s/step:40.168734323233366 - timing_s/stop_profile:0.00010610558092594147 - timing_per_token_ms/update_actor:0.12306077442790617 - timing_per_token_ms/gen:0.1417844530553678 - timing_per_token_ms/adv:0.003052558497384221 - perf/total_num_tokens:175031 - perf/time_per_step:40.168734323233366 - perf/throughput:544.6742440013945 (TaskRunner pid=2255397) Training Progress: 29%|██▉ | 29/100 [25:41<49:19, 41.68s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and identify the key functional groups and substituents that must be present in the reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key features: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar heterocycle). (TaskRunner pid=2255397) - A benzene ring with a chlorine substituent. (TaskRunner pid=2255397) - A trifluoromethyl group (C(F)(F)F) attached to the benzene ring. (TaskRunner pid=2255397) - A substituent group (likely a nitrogen-containing chain) that connects to the ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's examine each option: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A:** (TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) This option contains the trifluoromethyl group (C(F)(F)F) and the chlorinated benzene ring. It also contains a nitrogen-containing chain and a ring structure similar to the product. This option closely matches the structure of the product and is a strong candidate. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B:** (TaskRunner pid=2255397) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) This option has a chlorine substituent on the ring, but it lacks the trifluoromethyl group and the correct nitrogen-containing chain. It also has a different ring structure than the product. Therefore, it is not the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C:** (TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) This option contains the trifluoromethyl group and the chlorinated benzene ring. However, the structure is more complex and includes an oxygen-containing side chain that is not present in the product. It introduces an unnecessary oxygen group, making it incompatible with the product structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D:** (TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) This option contains the trifluoromethyl group and a benzene ring. However, it lacks the correct nitrogen-containing chain and the chlorinated benzene ring. The structure is not aligned with the product, so it is not the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion:** (TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and substituents present in the product, making it the only correct reactant for the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2255397) step:30 - global_seqlen/min:18031 - global_seqlen/max:24561 - global_seqlen/minmax_diff:6530 - global_seqlen/balanced_min:20880 - global_seqlen/balanced_max:20888 - global_seqlen/mean:20884.625 - actor/entropy:0.4658919870853424 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3171748220920563 - training/rollout_probs_diff_mean:0.005262547638267279 - training/rollout_probs_diff_std:0.01141402218490839 - training/rollout_actor_probs_pearson_corr:0.9991620779037476 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.44921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:0.9999656677246094 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.412619844311848e-05 - rollout_corr/rollout_is_max:1.9128998517990112 - rollout_corr/rollout_is_min:0.2724512219429016 - rollout_corr/rollout_is_std:0.03873920068144798 - rollout_corr/rollout_is_eff_sample_size:0.9985014042044956 - rollout_corr/rollout_is_seq_mean:1.0000505447387695 - rollout_corr/rollout_is_seq_std:0.002504864940419793 - rollout_corr/rollout_is_seq_max:1.0067648887634277 - rollout_corr/rollout_is_seq_min:0.9914875626564026 - rollout_corr/rollout_is_seq_max_deviation:0.008512437343597412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6606428353115916) - rollout_corr/training_log_ppl:np.float64(0.49546970275696367) - rollout_corr/kl:np.float64(0.0007864780202773147) - rollout_corr/k3_kl:np.float64(0.000911510776063551) - rollout_corr/rollout_ppl:np.float64(1.659348614513874) - rollout_corr/rollout_log_ppl:np.float64(0.4946832267160062) - rollout_corr/log_ppl_diff:np.float64(0.0007864760409574956) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019896051089745015) - rollout_corr/log_ppl_diff_max:np.float64(0.009873658418655396) - rollout_corr/log_ppl_diff_min:np.float64(-0.007493495941162109) - rollout_corr/ppl_ratio:np.float64(1.0007900123018771) - rollout_corr/chi2_token:np.float64(0.002100849524140358) - rollout_corr/chi2_seq:np.float64(0.6980828407686204) - actor/pg_loss:np.float64(0.0044937689090147614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004526598731899867) - actor/ppo_kl:np.float64(2.1002829232230624e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44921875) - self_distillation/token_reweight_w_mean:np.float64(16658.25212058681) - self_distillation/token_reweight_w_std:np.float64(262596.2304097906) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002752567641437) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09863085992401466) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.3747030645608902) - perf/mfu/actor:np.float64(0.050935440136856504) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.77979278564453) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.606845238095238) - val-aux/sciknoweval/reward/std@16:np.float64(0.2138942013523591) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6934285714285715) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.16904326809342193) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.521195238095238) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1926339687939083) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6076809523809524) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21426439691535565) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7574380952380952) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11875313696867805) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4448476190476191) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15604155437402734) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6224000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16648090775602947) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8030238095238095) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07678824143757296) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.38081904761904767) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11615050215883854) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6278476190476191) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.11644569510751888) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8329428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04605969898285256) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3325333333333333) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07403573137391646) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6297142857142858) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.079464901369361) - val-aux/sciknoweval/score/mean@16:np.float64(0.606845238095238) - val-aux/sciknoweval/score/std@16:np.float64(0.2138942013523591) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6934285714285715) - val-aux/sciknoweval/score/best@2/std:np.float64(0.16904326809342193) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.521195238095238) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1926339687939083) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6076809523809524) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21426439691535565) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7574380952380952) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11875313696867805) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4448476190476191) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15604155437402734) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6224000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16648090775602947) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8030238095238095) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07678824143757296) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.38081904761904767) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11615050215883854) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6278476190476191) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.11644569510751888) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8329428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04605969898285256) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3325333333333333) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07403573137391646) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6297142857142858) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.079464901369361) - val-core/sciknoweval/acc/mean@16:np.float64(0.606845238095238) - val-aux/sciknoweval/acc/std@16:np.float64(0.2138942013523591) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6934285714285715) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.16904326809342193) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.521195238095238) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1926339687939083) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6076809523809524) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21426439691535565) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7574380952380952) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11875313696867805) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4448476190476191) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15604155437402734) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6224000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16648090775602947) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8030238095238095) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07678824143757296) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.38081904761904767) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11615050215883854) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6278476190476191) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.11644569510751888) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8329428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04605969898285256) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3325333333333333) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07403573137391646) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6297142857142858) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.079464901369361) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019352398812770844 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0019352398812770844 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:343.39453125 - response_length/max:941.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.39453125 - response_length_non_aborted/max:941.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015097111463546753 - timing_s/agent_loop/generate_sequences/min:np.float64(1.869465732946992) - timing_s/agent_loop/generate_sequences/max:np.float64(7.8817349802702665) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.971169372940494) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.8817349802702665) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:941 - timing_s/gen:15.309523962438107 - timing_s/reward:0.06142178550362587 - timing_s/old_log_prob:3.0037817489355803 - timing_s/adv:0.5514516998082399 - timing_s/update_actor:21.344387443736196 - timing_s/step:40.35734134726226 - timing_s/testing:78.26906821876764 - timing_s/save_checkpoint:13.220651166513562 - timing_s/stop_profile:0.00014671683311462402 - timing_per_token_ms/update_actor:0.12775179973147827 - timing_per_token_ms/gen:0.17415195216005308 - timing_per_token_ms/adv:0.0033005841606459294 - perf/total_num_tokens:167077 - perf/time_per_step:40.35734134726226 - perf/throughput:517.4925875392621 (TaskRunner pid=2255397) Training Progress: 30%|███ | 30/100 [27:53<1:20:12, 68.76s/it] (TaskRunner pid=2255397) step:31 - global_seqlen/min:19249 - global_seqlen/max:25029 - global_seqlen/minmax_diff:5780 - global_seqlen/balanced_min:21607 - global_seqlen/balanced_max:21619 - global_seqlen/mean:21615.375 - actor/entropy:0.4569137990474701 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.303112655878067 - training/rollout_probs_diff_mean:0.004967319313436747 - training/rollout_probs_diff_std:0.010901066474616528 - training/rollout_actor_probs_pearson_corr:0.9992502331733704 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000017762184143 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.816590428352356 - rollout_corr/rollout_is_min:0.5354147553443909 - rollout_corr/rollout_is_std:0.03712382912635803 - rollout_corr/rollout_is_eff_sample_size:0.9986238370470686 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.002106655156239867 - rollout_corr/rollout_is_seq_max:1.0081474781036377 - rollout_corr/rollout_is_seq_min:0.9948439002037048 - rollout_corr/rollout_is_seq_max_deviation:0.008147478103637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6235803924500942) - rollout_corr/training_log_ppl:np.float64(0.4749105938244611) - rollout_corr/kl:np.float64(0.0008353461136003659) - rollout_corr/k3_kl:np.float64(0.00080985213742224) - rollout_corr/rollout_ppl:np.float64(1.6222249972634017) - rollout_corr/rollout_log_ppl:np.float64(0.4740752446523402) - rollout_corr/log_ppl_diff:np.float64(0.0008353491721209139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017292899719905108) - rollout_corr/log_ppl_diff_max:np.float64(0.006912916898727417) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048788487911224365) - rollout_corr/ppl_ratio:np.float64(1.0008378000929952) - rollout_corr/chi2_token:np.float64(0.0016067922115325928) - rollout_corr/chi2_seq:np.float64(0.6984529327601194) - actor/pg_loss:np.float64(0.005961017799563706) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028992991656195954) - actor/ppo_kl:np.float64(0.0001396356336407223) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(9105.852561026346) - self_distillation/token_reweight_w_std:np.float64(160757.80174179428) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988879463635385) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1225835490913596) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.33824220299720764) - perf/mfu/actor:np.float64(0.05203885094538403) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.26508331298828) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001895678462460637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001895678462460637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.63671875 - response_length/max:1088.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.63671875 - response_length_non_aborted/max:1088.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.687268614768982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6187728364020586) - timing_s/agent_loop/generate_sequences/max:np.float64(9.020029896870255) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.584768897177128) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.020029896870255) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:194 - timing_s/agent_loop/slowest/response_length:1088 - timing_s/gen:15.492790538817644 - timing_s/reward:0.05757281742990017 - timing_s/old_log_prob:3.137273196130991 - timing_s/adv:0.5795937422662973 - timing_s/update_actor:21.805233089253306 - timing_s/step:41.175627982243896 - timing_s/stop_profile:0.00012617185711860657 - timing_per_token_ms/update_actor:0.12609793427857086 - timing_per_token_ms/gen:0.15296535982166448 - timing_per_token_ms/adv:0.00335174466245842 - perf/total_num_tokens:172923 - perf/time_per_step:41.175627982243896 - perf/throughput:524.9555637456499 (TaskRunner pid=2255397) Training Progress: 31%|███ | 31/100 [28:34<1:09:34, 60.50s/it] (TaskRunner pid=2255397) step:32 - global_seqlen/min:17172 - global_seqlen/max:25152 - global_seqlen/minmax_diff:7980 - global_seqlen/balanced_min:20878 - global_seqlen/balanced_max:20886 - global_seqlen/mean:20883.0 - actor/entropy:0.4830181300640106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3013932406902313 - training/rollout_probs_diff_mean:0.00495893694460392 - training/rollout_probs_diff_std:0.010702594183385372 - training/rollout_actor_probs_pearson_corr:0.9992987513542175 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001959800720215 - rollout_corr/rollout_is_ratio_fraction_high:1.0022450624092016e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0067350962781347e-05 - rollout_corr/rollout_is_max:2.244924545288086 - rollout_corr/rollout_is_min:0.39147886633872986 - rollout_corr/rollout_is_std:0.037390995770692825 - rollout_corr/rollout_is_eff_sample_size:0.998604459752836 - rollout_corr/rollout_is_seq_mean:1.0002524852752686 - rollout_corr/rollout_is_seq_std:0.0021919296123087406 - rollout_corr/rollout_is_seq_max:1.0080853700637817 - rollout_corr/rollout_is_seq_min:0.9943554401397705 - rollout_corr/rollout_is_seq_max_deviation:0.008085370063781738 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6529409568756819) - rollout_corr/training_log_ppl:np.float64(0.4933903770870529) - rollout_corr/kl:np.float64(0.0006465979992356097) - rollout_corr/k3_kl:np.float64(0.0008415120931317688) - rollout_corr/rollout_ppl:np.float64(1.651906920131296) - rollout_corr/rollout_log_ppl:np.float64(0.49274377603433095) - rollout_corr/log_ppl_diff:np.float64(0.0006466010527219623) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017735535220708698) - rollout_corr/log_ppl_diff_max:np.float64(0.007363229990005493) - rollout_corr/log_ppl_diff_min:np.float64(-0.010225236415863037) - rollout_corr/ppl_ratio:np.float64(1.000649468274787) - rollout_corr/chi2_token:np.float64(0.0021671864669770002) - rollout_corr/chi2_seq:np.float64(1.2125774417072535) - actor/pg_loss:np.float64(0.007539922837167978) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005248926790955011) - actor/ppo_kl:np.float64(0.00015378757747441796) - actor/pg_clipfrac_lower:np.float64(1.2849506674683653e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(28962.190404822817) - self_distillation/token_reweight_w_std:np.float64(594865.4367179495) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994088341481984) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15152001177193597) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4039623737335205) - perf/mfu/actor:np.float64(0.050354520500870484) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.79799795150757) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005901970434933901 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005901970434933901 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:389.75 - response_length/max:1026.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:389.75 - response_length_non_aborted/max:1026.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001097116619348526 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7604525182396173) - timing_s/agent_loop/generate_sequences/max:np.float64(8.474037919193506) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.519642143648525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.474037919193506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:1026 - timing_s/gen:15.50579597428441 - timing_s/reward:0.05374091491103172 - timing_s/old_log_prob:3.0371324717998505 - timing_s/adv:0.5603868458420038 - timing_s/update_actor:21.465026825666428 - timing_s/step:40.65800975076854 - timing_s/stop_profile:0.0001172088086605072 - timing_per_token_ms/update_actor:0.12848385544262336 - timing_per_token_ms/gen:0.1554060693381616 - timing_per_token_ms/adv:0.0033543243657640416 - perf/total_num_tokens:167064 - perf/time_per_step:40.65800975076854 - perf/throughput:513.6257315105114 (TaskRunner pid=2255397) Training Progress: 32%|███▏ | 32/100 [29:15<1:01:50, 54.56s/it] (TaskRunner pid=2255397) step:33 - global_seqlen/min:16273 - global_seqlen/max:24765 - global_seqlen/minmax_diff:8492 - global_seqlen/balanced_min:20358 - global_seqlen/balanced_max:20368 - global_seqlen/mean:20365.75 - actor/entropy:0.5308820605278015 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2607361674308777 - training/rollout_probs_diff_mean:0.005311037413775921 - training/rollout_probs_diff_std:0.010893904604017735 - training/rollout_actor_probs_pearson_corr:0.9993165731430054 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:2.2798258214606903e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1399129107303452e-05 - rollout_corr/rollout_is_max:2.4295754432678223 - rollout_corr/rollout_is_min:0.4068605899810791 - rollout_corr/rollout_is_std:0.038521647453308105 - rollout_corr/rollout_is_eff_sample_size:0.9985184003675729 - rollout_corr/rollout_is_seq_mean:1.0000193119049072 - rollout_corr/rollout_is_seq_std:0.0023478458169847727 - rollout_corr/rollout_is_seq_max:1.0072718858718872 - rollout_corr/rollout_is_seq_min:0.9926888346672058 - rollout_corr/rollout_is_seq_max_deviation:0.0073111653327941895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7451868052594364) - rollout_corr/training_log_ppl:np.float64(0.5472010709927417) - rollout_corr/kl:np.float64(0.000750623744014689) - rollout_corr/k3_kl:np.float64(0.0009170059764755933) - rollout_corr/rollout_ppl:np.float64(1.7438560076989233) - rollout_corr/rollout_log_ppl:np.float64(0.5464504445553757) - rollout_corr/log_ppl_diff:np.float64(0.0007506264373660088) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020804237574338913) - rollout_corr/log_ppl_diff_max:np.float64(0.011185050010681152) - rollout_corr/log_ppl_diff_min:np.float64(-0.008495688438415527) - rollout_corr/ppl_ratio:np.float64(1.0007542204111814) - rollout_corr/chi2_token:np.float64(0.0021686023101210594) - rollout_corr/chi2_seq:np.float64(1.0599001871887594) - actor/pg_loss:np.float64(0.0046818595146760345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000626342507530353) - actor/ppo_kl:np.float64(-7.711236621688045e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(406.9383929811884) - self_distillation/token_reweight_w_std:np.float64(6888.8986393570085) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006252727471292) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10766920002060942) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.35059650987386703) - perf/mfu/actor:np.float64(0.04906215016751242) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.48283004760742) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038529057055711746 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038529057055711746 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:342.6796875 - response_length/max:820.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:342.6796875 - response_length_non_aborted/max:820.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011102668941020966 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7211053427308798) - timing_s/agent_loop/generate_sequences/max:np.float64(6.794723700731993) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.86705601822905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.794723700731993) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:378 - timing_s/agent_loop/slowest/response_length:820 - timing_s/gen:13.564623160287738 - timing_s/reward:0.06091204658150673 - timing_s/old_log_prob:2.9387052599340677 - timing_s/adv:0.5545471422374249 - timing_s/update_actor:21.675176206976175 - timing_s/step:38.87970031797886 - timing_s/stop_profile:3.417767584323883e-05 - timing_per_token_ms/update_actor:0.1330369382847193 - timing_per_token_ms/gen:0.15462489068563182 - timing_per_token_ms/adv:0.0034036749336350544 - perf/total_num_tokens:162926 - perf/time_per_step:38.87970031797886 - perf/throughput:523.8144798812251 (TaskRunner pid=2255397) Training Progress: 33%|███▎ | 33/100 [29:54<55:40, 49.86s/it] (TaskRunner pid=2255397) step:34 - global_seqlen/min:19407 - global_seqlen/max:28954 - global_seqlen/minmax_diff:9547 - global_seqlen/balanced_min:20764 - global_seqlen/balanced_max:28104 - global_seqlen/mean:21689.625 - actor/entropy:0.4711659848690033 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26104798913002014 - training/rollout_probs_diff_mean:0.004729302600026131 - training/rollout_probs_diff_std:0.010431492701172829 - training/rollout_actor_probs_pearson_corr:0.9993188977241516 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9998348355293274 - rollout_corr/rollout_is_ratio_fraction_high:1.0172008842346258e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.372540235519409 - rollout_corr/rollout_is_min:0.5585512518882751 - rollout_corr/rollout_is_std:0.03565623611211777 - rollout_corr/rollout_is_eff_sample_size:0.9987297716220098 - rollout_corr/rollout_is_seq_mean:0.9998084902763367 - rollout_corr/rollout_is_seq_std:0.0021642311476171017 - rollout_corr/rollout_is_seq_max:1.0065914392471313 - rollout_corr/rollout_is_seq_min:0.9937243461608887 - rollout_corr/rollout_is_seq_max_deviation:0.006591439247131348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.732406364288181) - rollout_corr/training_log_ppl:np.float64(0.53354305683024) - rollout_corr/kl:np.float64(0.0010742717632254706) - rollout_corr/k3_kl:np.float64(0.0007844316083662761) - rollout_corr/rollout_ppl:np.float64(1.7304212134331465) - rollout_corr/rollout_log_ppl:np.float64(0.5324687818174425) - rollout_corr/log_ppl_diff:np.float64(0.0010742750127974432) - rollout_corr/log_ppl_abs_diff:np.float64(0.002004527148528723) - rollout_corr/log_ppl_diff_max:np.float64(0.010212719440460205) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042626261711120605) - rollout_corr/ppl_ratio:np.float64(1.0010776109993458) - rollout_corr/chi2_token:np.float64(0.000989705789834261) - rollout_corr/chi2_seq:np.float64(0.5540260763373226) - actor/pg_loss:np.float64(0.005981583381071687) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004929028409605962) - actor/ppo_kl:np.float64(0.00015171869483765477) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(11402.142899290891) - self_distillation/token_reweight_w_std:np.float64(223975.93910462921) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007735511753708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13933006773004308) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.3575190082192421) - perf/mfu/actor:np.float64(0.04880687442998337) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.40957641601562) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05439735949039459 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05439735949039459 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.01953125 - response_length/max:8192.0 - response_length/min:160.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:384.01953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010099261999130249 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5427038352936506) - timing_s/agent_loop/generate_sequences/max:np.float64(57.38408918492496) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.160430314703262) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.38408918492496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:438 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.30811233632267 - timing_s/reward:0.06254505552351475 - timing_s/old_log_prob:3.2206109445542097 - timing_s/adv:0.5401895362883806 - timing_s/update_actor:23.241615699604154 - timing_s/step:91.45890662260354 - timing_s/stop_profile:2.9904767870903015e-05 - timing_per_token_ms/update_actor:0.1339443149639756 - timing_per_token_ms/gen:0.6541426760146342 - timing_per_token_ms/adv:0.003113179321267545 - perf/total_num_tokens:173517 - perf/time_per_step:91.45890662260354 - perf/throughput:237.15158863094845 (TaskRunner pid=2255397) Training Progress: 34%|███▍ | 34/100 [31:25<1:08:35, 62.35s/it] (TaskRunner pid=2255397) step:35 - global_seqlen/min:19463 - global_seqlen/max:21625 - global_seqlen/minmax_diff:2162 - global_seqlen/balanced_min:20560 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20569.125 - actor/entropy:0.4701376259326935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27064740657806396 - training/rollout_probs_diff_mean:0.00488562835380435 - training/rollout_probs_diff_std:0.010647308081388474 - training/rollout_actor_probs_pearson_corr:0.999289870262146 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:0.9999282956123352 - rollout_corr/rollout_is_ratio_fraction_high:1.0713864867284428e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1437816619873047 - rollout_corr/rollout_is_min:0.5562102794647217 - rollout_corr/rollout_is_std:0.036710504442453384 - rollout_corr/rollout_is_eff_sample_size:0.9986539149503298 - rollout_corr/rollout_is_seq_mean:0.9999589323997498 - rollout_corr/rollout_is_seq_std:0.0022408063523471355 - rollout_corr/rollout_is_seq_max:1.0093075037002563 - rollout_corr/rollout_is_seq_min:0.9939238429069519 - rollout_corr/rollout_is_seq_max_deviation:0.009307503700256348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.643273327499628) - rollout_corr/training_log_ppl:np.float64(0.48507256753509864) - rollout_corr/kl:np.float64(0.0008028178892658389) - rollout_corr/k3_kl:np.float64(0.0007829796426221947) - rollout_corr/rollout_ppl:np.float64(1.641896395944059) - rollout_corr/rollout_log_ppl:np.float64(0.4842697436106391) - rollout_corr/log_ppl_diff:np.float64(0.000802823924459517) - rollout_corr/log_ppl_abs_diff:np.float64(0.002036678488366306) - rollout_corr/log_ppl_diff_max:np.float64(0.008617639541625977) - rollout_corr/log_ppl_diff_min:np.float64(-0.010091245174407959) - rollout_corr/ppl_ratio:np.float64(1.0008062841370702) - rollout_corr/chi2_token:np.float64(0.0015425898600369692) - rollout_corr/chi2_seq:np.float64(1.5650525961536914) - actor/pg_loss:np.float64(0.0024570953100919724) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004893736586382147) - actor/ppo_kl:np.float64(4.371891989851662e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(28164.23297266336) - self_distillation/token_reweight_w_std:np.float64(438345.5305433142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0024336164351553) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09937885942053981) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2858409211039543) - perf/mfu/actor:np.float64(0.04990645011261699) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.3169288635254) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004351168405264616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004351168405264616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:364.59765625 - response_length/max:846.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:364.59765625 - response_length_non_aborted/max:846.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.146092295646667e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5682779010385275) - timing_s/agent_loop/generate_sequences/max:np.float64(7.532088892534375) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.356410696163948) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.532088892534375) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:846 - timing_s/gen:14.197907466441393 - timing_s/reward:0.05897785350680351 - timing_s/old_log_prob:3.0003139451146126 - timing_s/adv:0.5386089980602264 - timing_s/update_actor:21.315383506938815 - timing_s/step:39.194119876250625 - timing_s/stop_profile:0.00012629106640815735 - timing_per_token_ms/update_actor:0.12953506473257137 - timing_per_token_ms/gen:0.15211446121518146 - timing_per_token_ms/adv:0.003273164257474652 - perf/total_num_tokens:164553 - perf/time_per_step:39.194119876250625 - perf/throughput:524.8012983821 (TaskRunner pid=2255397) Training Progress: 35%|███▌ | 35/100 [32:05<1:00:02, 55.42s/it] (TaskRunner pid=2255397) step:36 - global_seqlen/min:15946 - global_seqlen/max:23368 - global_seqlen/minmax_diff:7422 - global_seqlen/balanced_min:21144 - global_seqlen/balanced_max:21148 - global_seqlen/mean:21146.25 - actor/entropy:0.4903087913990021 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462920188903809 - training/rollout_probs_diff_mean:0.005082453601062298 - training/rollout_probs_diff_std:0.010714489035308361 - training/rollout_actor_probs_pearson_corr:0.9993000030517578 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0000641345977783 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6886519193649292 - rollout_corr/rollout_is_min:0.6301243901252747 - rollout_corr/rollout_is_std:0.037293627858161926 - rollout_corr/rollout_is_eff_sample_size:0.9986113546547832 - rollout_corr/rollout_is_seq_mean:1.000097632408142 - rollout_corr/rollout_is_seq_std:0.002074406947940588 - rollout_corr/rollout_is_seq_max:1.0062254667282104 - rollout_corr/rollout_is_seq_min:0.9938981533050537 - rollout_corr/rollout_is_seq_max_deviation:0.006225466728210449 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6528221243061125) - rollout_corr/training_log_ppl:np.float64(0.4915998738142662) - rollout_corr/kl:np.float64(0.0008901074607035042) - rollout_corr/k3_kl:np.float64(0.000780510345350649) - rollout_corr/rollout_ppl:np.float64(1.6513647134415805) - rollout_corr/rollout_log_ppl:np.float64(0.49070976028451696) - rollout_corr/log_ppl_diff:np.float64(0.0008901135297492146) - rollout_corr/log_ppl_abs_diff:np.float64(0.001717470120638609) - rollout_corr/log_ppl_diff_max:np.float64(0.006238460540771484) - rollout_corr/log_ppl_diff_min:np.float64(-0.005200505256652832) - rollout_corr/ppl_ratio:np.float64(1.0008924398571253) - rollout_corr/chi2_token:np.float64(0.0013305631000548601) - rollout_corr/chi2_seq:np.float64(0.19096036697737873) - actor/pg_loss:np.float64(0.0061755290953442454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007742183429400029) - actor/ppo_kl:np.float64(0.0002702638142206837) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(5297.740719528403) - self_distillation/token_reweight_w_std:np.float64(105322.54200576013) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011047541629523) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1423137153033167) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.3517342582345009) - perf/mfu/actor:np.float64(0.051460493101783154) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.31007385253906) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027716271579265594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027716271579265594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.8203125 - response_length/max:718.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.8203125 - response_length_non_aborted/max:718.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014107488095760345 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2529861833900213) - timing_s/agent_loop/generate_sequences/max:np.float64(7.170260231941938) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.674387803606805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.170260231941938) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:682 - timing_s/gen:14.726460101082921 - timing_s/reward:0.061883002519607544 - timing_s/old_log_prob:2.9912548549473286 - timing_s/adv:0.5396729540079832 - timing_s/update_actor:21.34972732514143 - timing_s/step:39.758028803393245 - timing_s/stop_profile:0.0001045670360326767 - timing_per_token_ms/update_actor:0.12620279792600006 - timing_per_token_ms/gen:0.15026693435932859 - timing_per_token_ms/adv:0.0031901220902523096 - perf/total_num_tokens:169170 - perf/time_per_step:39.758028803393245 - perf/throughput:531.8737029084103 (TaskRunner pid=2255397) Training Progress: 36%|███▌ | 36/100 [32:44<54:07, 50.74s/it] (TaskRunner pid=2255397) step:37 - global_seqlen/min:18794 - global_seqlen/max:24933 - global_seqlen/minmax_diff:6139 - global_seqlen/balanced_min:22097 - global_seqlen/balanced_max:22101 - global_seqlen/mean:22100.0 - actor/entropy:0.4772471785545349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8962392210960388 - training/rollout_probs_diff_mean:0.004988708533346653 - training/rollout_probs_diff_std:0.011156635358929634 - training/rollout_actor_probs_pearson_corr:0.9992755651473999 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999436140060425 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.1014806154416874e-05 - rollout_corr/rollout_is_max:1.5039196014404297 - rollout_corr/rollout_is_min:0.0692535936832428 - rollout_corr/rollout_is_std:0.03678025305271149 - rollout_corr/rollout_is_eff_sample_size:0.9986488027678234 - rollout_corr/rollout_is_seq_mean:0.9999807476997375 - rollout_corr/rollout_is_seq_std:0.002165940823033452 - rollout_corr/rollout_is_seq_max:1.0075201988220215 - rollout_corr/rollout_is_seq_min:0.9936956763267517 - rollout_corr/rollout_is_seq_max_deviation:0.007520198822021484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6701486743986607) - rollout_corr/training_log_ppl:np.float64(0.5012833332875744) - rollout_corr/kl:np.float64(0.0008904281232275935) - rollout_corr/k3_kl:np.float64(0.0009389091193838794) - rollout_corr/rollout_ppl:np.float64(1.6685718595981598) - rollout_corr/rollout_log_ppl:np.float64(0.5003929063677788) - rollout_corr/log_ppl_diff:np.float64(0.0008904269197955728) - rollout_corr/log_ppl_abs_diff:np.float64(0.001979760592803359) - rollout_corr/log_ppl_diff_max:np.float64(0.02197730541229248) - rollout_corr/log_ppl_diff_min:np.float64(-0.00723642110824585) - rollout_corr/ppl_ratio:np.float64(1.0008947688620538) - rollout_corr/chi2_token:np.float64(0.0017904054839164019) - rollout_corr/chi2_seq:np.float64(0.7441312682349235) - actor/pg_loss:np.float64(0.0051974470261484385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006028023444741848) - actor/ppo_kl:np.float64(0.00011284644474807148) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(17486.880942083197) - self_distillation/token_reweight_w_std:np.float64(248698.93032729038) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008525024168193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10978465108200908) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.33661027252674103) - perf/mfu/actor:np.float64(0.053537470413758295) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.20091247558594) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00370756140910089 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00370756140910089 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.84375 - response_length/max:858.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.84375 - response_length_non_aborted/max:858.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010385364294052124 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9452894497662783) - timing_s/agent_loop/generate_sequences/max:np.float64(7.737249631434679) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.423385596266598) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.737249631434679) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:858 - timing_s/gen:14.69729588367045 - timing_s/reward:0.05707615241408348 - timing_s/old_log_prob:3.1259428039193153 - timing_s/adv:0.5428317990154028 - timing_s/update_actor:21.357854707166553 - timing_s/step:39.88826394826174 - timing_s/stop_profile:0.00012905150651931763 - timing_per_token_ms/update_actor:0.1208023456287701 - timing_per_token_ms/gen:0.1519445856801593 - timing_per_token_ms/adv:0.00307031560529074 - perf/total_num_tokens:176800 - perf/time_per_step:39.88826394826174 - perf/throughput:554.0476774989622 (TaskRunner pid=2255397) Training Progress: 37%|███▋ | 37/100 [33:24<49:52, 47.50s/it] (TaskRunner pid=2255397) step:38 - global_seqlen/min:17401 - global_seqlen/max:23183 - global_seqlen/minmax_diff:5782 - global_seqlen/balanced_min:20614 - global_seqlen/balanced_max:20619 - global_seqlen/mean:20617.125 - actor/entropy:0.4591502249240875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21379658579826355 - training/rollout_probs_diff_mean:0.005224332213401794 - training/rollout_probs_diff_std:0.011259016580879688 - training/rollout_actor_probs_pearson_corr:0.9992035627365112 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001171827316284 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0498313460848294e-05 - rollout_corr/rollout_is_max:1.9823302030563354 - rollout_corr/rollout_is_min:0.3327290415763855 - rollout_corr/rollout_is_std:0.03824835270643234 - rollout_corr/rollout_is_eff_sample_size:0.9985396761004448 - rollout_corr/rollout_is_seq_mean:1.0001174211502075 - rollout_corr/rollout_is_seq_std:0.002267003059387207 - rollout_corr/rollout_is_seq_max:1.007875919342041 - rollout_corr/rollout_is_seq_min:0.9941402077674866 - rollout_corr/rollout_is_seq_max_deviation:0.007875919342041016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.597476608119905) - rollout_corr/training_log_ppl:np.float64(0.4596842197352089) - rollout_corr/kl:np.float64(0.0007120158846873892) - rollout_corr/k3_kl:np.float64(0.0008242313452342387) - rollout_corr/rollout_ppl:np.float64(1.5963355526328087) - rollout_corr/rollout_log_ppl:np.float64(0.45897220366168767) - rollout_corr/log_ppl_diff:np.float64(0.0007120160735212266) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018952106474898756) - rollout_corr/log_ppl_diff_max:np.float64(0.006997823715209961) - rollout_corr/log_ppl_diff_min:np.float64(-0.006651520729064941) - rollout_corr/ppl_ratio:np.float64(1.0007150201126933) - rollout_corr/chi2_token:np.float64(0.001860827673226595) - rollout_corr/chi2_seq:np.float64(0.6607308345846832) - actor/pg_loss:np.float64(0.003047003992833197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027509686378834886) - actor/ppo_kl:np.float64(6.150551095629453e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(24324.67800112092) - self_distillation/token_reweight_w_std:np.float64(319731.90878068085) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001283828169107) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07227780664106831) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.25013007409870625) - perf/mfu/actor:np.float64(0.05036454480977212) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.1811408996582) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011094711953774095 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011094711953774095 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:381.12890625 - response_length/max:764.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.12890625 - response_length_non_aborted/max:764.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010602176189422607 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6879636067897081) - timing_s/agent_loop/generate_sequences/max:np.float64(6.8860987443476915) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.44739578160079) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.8860987443476915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:335 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:14.023487392812967 - timing_s/reward:0.054636452347040176 - timing_s/old_log_prob:2.9677660800516605 - timing_s/adv:0.5456443149596453 - timing_s/update_actor:21.413069240748882 - timing_s/step:39.11168333515525 - timing_s/stop_profile:0.00010557286441326141 - timing_per_token_ms/update_actor:0.1298257470473507 - timing_per_token_ms/gen:0.14372892407232796 - timing_per_token_ms/adv:0.003308198372467338 - perf/total_num_tokens:164937 - perf/time_per_step:39.11168333515525 - perf/throughput:527.1346882037278 (TaskRunner pid=2255397) Training Progress: 38%|███▊ | 38/100 [34:04<46:30, 45.01s/it] (TaskRunner pid=2255397) step:39 - global_seqlen/min:16098 - global_seqlen/max:24192 - global_seqlen/minmax_diff:8094 - global_seqlen/balanced_min:21469 - global_seqlen/balanced_max:21484 - global_seqlen/mean:21479.5 - actor/entropy:0.4425225853919983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17422929406166077 - training/rollout_probs_diff_mean:0.004969022236764431 - training/rollout_probs_diff_std:0.010954358614981174 - training/rollout_actor_probs_pearson_corr:0.9992186427116394 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0003507137298584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6815071105957031 - rollout_corr/rollout_is_min:0.5040706396102905 - rollout_corr/rollout_is_std:0.03746902570128441 - rollout_corr/rollout_is_eff_sample_size:0.9985991103255378 - rollout_corr/rollout_is_seq_mean:1.0004005432128906 - rollout_corr/rollout_is_seq_std:0.0023794230073690414 - rollout_corr/rollout_is_seq_max:1.012694001197815 - rollout_corr/rollout_is_seq_min:0.9931012392044067 - rollout_corr/rollout_is_seq_max_deviation:0.012694001197814941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6068592290394008) - rollout_corr/training_log_ppl:np.float64(0.4637760063924361) - rollout_corr/kl:np.float64(0.0005125066249966892) - rollout_corr/k3_kl:np.float64(0.0008192024238837803) - rollout_corr/rollout_ppl:np.float64(1.6059974883683026) - rollout_corr/rollout_log_ppl:np.float64(0.46326349792070687) - rollout_corr/log_ppl_diff:np.float64(0.0005125084717292339) - rollout_corr/log_ppl_abs_diff:np.float64(0.001737583806971088) - rollout_corr/log_ppl_diff_max:np.float64(0.006271004676818848) - rollout_corr/log_ppl_diff_min:np.float64(-0.01115313172340393) - rollout_corr/ppl_ratio:np.float64(1.0005151890218258) - rollout_corr/chi2_token:np.float64(0.0022532844450324774) - rollout_corr/chi2_seq:np.float64(1.0010677098762244) - actor/pg_loss:np.float64(0.004201453877612948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00037899658491369337) - actor/ppo_kl:np.float64(0.000158508868715046) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(21191.4891625538) - self_distillation/token_reweight_w_std:np.float64(305819.6095345903) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013274641241878) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08719093064428307) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.2567609027028084) - perf/mfu/actor:np.float64(0.05189741811293595) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.25640106201172) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00024856385425664485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00024856385425664485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.953125 - response_length/max:1077.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:388.953125 - response_length_non_aborted/max:1077.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001035984605550766 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4913760963827372) - timing_s/agent_loop/generate_sequences/max:np.float64(8.839037172496319) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.348892316258571) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.839037172496319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1077 - timing_s/gen:15.795821461826563 - timing_s/reward:0.061745528131723404 - timing_s/old_log_prob:3.0005341172218323 - timing_s/adv:0.5373612400144339 - timing_s/update_actor:21.613999467343092 - timing_s/step:41.096161579713225 - timing_s/stop_profile:2.9422342777252197e-05 - timing_per_token_ms/update_actor:0.12578271996172566 - timing_per_token_ms/gen:0.15863718175618208 - timing_per_token_ms/adv:0.0031271749808796405 - perf/total_num_tokens:171836 - perf/time_per_step:41.096161579713225 - perf/throughput:522.6643845639145 (TaskRunner pid=2255397) Training Progress: 39%|███▉ | 39/100 [34:45<44:34, 43.84s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must examine the structure of the product and compare it to the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains a nitrogen-containing ring with a substituent group that includes a chlorinated benzene ring and a trifluoromethyl group. Specifically, the product has a structure where a nitrogen atom is part of a ring that is connected to a carbon chain leading to a chlorinated benzene ring with a trifluoromethyl group. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's analyze the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option A**: Contains a structure with a nitrogen ring and a chlorinated benzene ring, along with a trifluoromethyl group. It closely resembles the structure of the target product, especially in terms of the nitrogen-containing ring and the substituent groups. (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option B**: Contains a chlorine atom directly attached to the nitrogen ring, which is not present in the target product. This makes it an incorrect match. (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option C**: Features a hydroxyl group and a different substituent pattern, which does not match the target structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option D**: Includes a double bond and a fluorinated carbon chain, but not the specific chlorinated benzene ring and nitrogen ring arrangement of the target product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Only **Option A** contains the correct structural elements that match the target product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2255397) step:40 - global_seqlen/min:18616 - global_seqlen/max:26413 - global_seqlen/minmax_diff:7797 - global_seqlen/balanced_min:22648 - global_seqlen/balanced_max:22652 - global_seqlen/mean:22651.25 - actor/entropy:0.46682530641555786 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102363586425781 - training/rollout_probs_diff_mean:0.005161741282790899 - training/rollout_probs_diff_std:0.011252960190176964 - training/rollout_actor_probs_pearson_corr:0.9991944432258606 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.0001730918884277 - rollout_corr/rollout_is_ratio_fraction_high:9.88513511401834e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.977027022803668e-05 - rollout_corr/rollout_is_max:2.2787070274353027 - rollout_corr/rollout_is_min:0.2646176815032959 - rollout_corr/rollout_is_std:0.03861745819449425 - rollout_corr/rollout_is_eff_sample_size:0.9985113878945102 - rollout_corr/rollout_is_seq_mean:1.0001778602600098 - rollout_corr/rollout_is_seq_std:0.0021594883874058723 - rollout_corr/rollout_is_seq_max:1.0066732168197632 - rollout_corr/rollout_is_seq_min:0.9935801029205322 - rollout_corr/rollout_is_seq_max_deviation:0.006673216819763184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6297845151275396) - rollout_corr/training_log_ppl:np.float64(0.4774156858911738) - rollout_corr/kl:np.float64(0.0007412694044468537) - rollout_corr/k3_kl:np.float64(0.0009004596647628205) - rollout_corr/rollout_ppl:np.float64(1.628512756433338) - rollout_corr/rollout_log_ppl:np.float64(0.47667441627709195) - rollout_corr/log_ppl_diff:np.float64(0.0007412696140818298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018867639009840786) - rollout_corr/log_ppl_diff_max:np.float64(0.00960838794708252) - rollout_corr/log_ppl_diff_min:np.float64(-0.005440711975097656) - rollout_corr/ppl_ratio:np.float64(1.0007440785411745) - rollout_corr/chi2_token:np.float64(0.0020869714207947254) - rollout_corr/chi2_seq:np.float64(0.9351563951931894) - actor/pg_loss:np.float64(0.003354526706971228) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000359561878212844) - actor/ppo_kl:np.float64(0.00013469158743095733) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(9349.600620643701) - self_distillation/token_reweight_w_std:np.float64(148328.15592557297) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9980465415865183) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08430611382937059) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.28926024958491325) - perf/mfu/actor:np.float64(0.054041057160877155) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.16180038452148) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6455357142857143) - val-aux/sciknoweval/reward/std@16:np.float64(0.19709042049186706) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7234142857142857) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15652759126874422) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5656380952380953) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17688873171951364) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6442142857142857) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19706989234623676) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7823238095238095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11006778616480029) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4971333333333333) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14506188764023242) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.657395238095238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15434237064022824) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8243714285714285) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07581488544967824) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43891904761904754) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10956011972160636) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6651571428571428) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10880726297391681) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8535380952380953) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04777075262896029) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3930904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0728366598950776) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6670523809523811) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07675932385019639) - val-aux/sciknoweval/score/mean@16:np.float64(0.6455357142857143) - val-aux/sciknoweval/score/std@16:np.float64(0.19709042049186706) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7234142857142857) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15652759126874422) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5656380952380953) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17688873171951364) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6442142857142857) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19706989234623676) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7823238095238095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11006778616480029) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4971333333333333) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14506188764023242) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.657395238095238) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15434237064022824) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8243714285714285) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07581488544967824) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43891904761904754) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10956011972160636) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6651571428571428) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10880726297391681) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8535380952380953) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04777075262896029) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3930904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0728366598950776) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6670523809523811) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07675932385019639) - val-core/sciknoweval/acc/mean@16:np.float64(0.6455357142857143) - val-aux/sciknoweval/acc/std@16:np.float64(0.19709042049186706) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7234142857142857) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15652759126874422) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5656380952380953) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17688873171951364) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6442142857142857) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19706989234623676) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7823238095238095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11006778616480029) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4971333333333333) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14506188764023242) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.657395238095238) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15434237064022824) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8243714285714285) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07581488544967824) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43891904761904754) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10956011972160636) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6651571428571428) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10880726297391681) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8535380952380953) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04777075262896029) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3930904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0728366598950776) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6670523809523811) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07675932385019639) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9970238095238095) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.008247511677748322) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.999504761904762) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0031118665720636365) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9944428571428572) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.010692005762772086) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.996947619047619) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.00845281646011912) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0002604287412166727) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9903238095238096) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.012632535914270398) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9988238095238094) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.005212831716796075) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9847809523809524) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.01316242814531538) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997285714285713) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.002375569550118018) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9783666666666666) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.010799831142838577) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0004863840055089889) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015742076793685555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015742076793685555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.1640625 - response_length/max:913.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.1640625 - response_length_non_aborted/max:913.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6823952794075012e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.958938129246235) - timing_s/agent_loop/generate_sequences/max:np.float64(8.071589579805732) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.574361211241921) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.071589579805732) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:913 - timing_s/gen:15.109674572944641 - timing_s/reward:0.06512491777539253 - timing_s/old_log_prob:3.012589380145073 - timing_s/adv:0.5300268922001123 - timing_s/update_actor:21.85186207294464 - timing_s/step:40.66220090165734 - timing_s/testing:138.401902012527 - timing_s/save_checkpoint:13.304814949631691 - timing_s/stop_profile:3.7167221307754517e-05 - timing_per_token_ms/update_actor:0.12058861030265791 - timing_per_token_ms/gen:0.1493611689462905 - timing_per_token_ms/adv:0.0029249318039849476 - perf/total_num_tokens:181210 - perf/time_per_step:40.66220090165734 - perf/throughput:557.0591236510458 (TaskRunner pid=2255397) Training Progress: 40%|████ | 40/100 [37:57<1:28:24, 88.41s/it] (TaskRunner pid=2255397) step:41 - global_seqlen/min:18283 - global_seqlen/max:24700 - global_seqlen/minmax_diff:6417 - global_seqlen/balanced_min:21101 - global_seqlen/balanced_max:21109 - global_seqlen/mean:21105.5 - actor/entropy:0.42351460456848145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9826107025146484 - training/rollout_probs_diff_mean:0.005102348513901234 - training/rollout_probs_diff_std:0.01174638606607914 - training/rollout_actor_probs_pearson_corr:0.9991093873977661 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.999843180179596 - rollout_corr/rollout_is_ratio_fraction_high:9.873228009382728e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.873228009382728e-06 - rollout_corr/rollout_is_max:2.636786937713623 - rollout_corr/rollout_is_min:0.013885053806006908 - rollout_corr/rollout_is_std:0.03809376060962677 - rollout_corr/rollout_is_eff_sample_size:0.9985506114667393 - rollout_corr/rollout_is_seq_mean:0.9998304843902588 - rollout_corr/rollout_is_seq_std:0.001992590492591262 - rollout_corr/rollout_is_seq_max:1.0068778991699219 - rollout_corr/rollout_is_seq_min:0.9926929473876953 - rollout_corr/rollout_is_seq_max_deviation:0.0073070526123046875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5631140493787825) - rollout_corr/training_log_ppl:np.float64(0.436920084990561) - rollout_corr/kl:np.float64(0.0011501897124975846) - rollout_corr/k3_kl:np.float64(0.0009579621572584074) - rollout_corr/rollout_ppl:np.float64(1.5612433003261685) - rollout_corr/rollout_log_ppl:np.float64(0.4357698948588222) - rollout_corr/log_ppl_diff:np.float64(0.001150190131738782) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019631539471447468) - rollout_corr/log_ppl_diff_max:np.float64(0.010622143745422363) - rollout_corr/log_ppl_diff_min:np.float64(-0.00589299201965332) - rollout_corr/ppl_ratio:np.float64(1.001153603894636) - rollout_corr/chi2_token:np.float64(0.0014891347382217646) - rollout_corr/chi2_seq:np.float64(0.3713759642560035) - actor/pg_loss:np.float64(0.007653480512090027) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008902258696252829) - actor/ppo_kl:np.float64(0.0001822680885723571) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(1181.0103185500484) - self_distillation/token_reweight_w_std:np.float64(19672.96590518701) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9963315855711699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1375392735353671) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.34661057591438293) - perf/mfu/actor:np.float64(0.05125395817020568) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.22666931152344) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004591051023453474 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004591051023453474 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.640625 - response_length/max:930.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.640625 - response_length_non_aborted/max:930.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7123838663101196e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.571682434529066) - timing_s/agent_loop/generate_sequences/max:np.float64(8.058521373197436) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.402195893351745) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.058521373197436) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:354 - timing_s/agent_loop/slowest/response_length:930 - timing_s/gen:14.408957185223699 - timing_s/reward:0.055850500240921974 - timing_s/old_log_prob:3.1053357888013124 - timing_s/adv:0.5568513758480549 - timing_s/update_actor:21.503483591601253 - timing_s/step:39.73492740653455 - timing_s/stop_profile:3.6053359508514404e-05 - timing_per_token_ms/update_actor:0.12735710828694685 - timing_per_token_ms/gen:0.1422629160106601 - timing_per_token_ms/adv:0.003298022884130054 - perf/total_num_tokens:168844 - perf/time_per_step:39.73492740653455 - perf/throughput:531.157381617089 (TaskRunner pid=2255397) Training Progress: 41%|████ | 41/100 [38:37<1:12:34, 73.81s/it] (TaskRunner pid=2255397) step:42 - global_seqlen/min:15988 - global_seqlen/max:22821 - global_seqlen/minmax_diff:6833 - global_seqlen/balanced_min:19797 - global_seqlen/balanced_max:19801 - global_seqlen/mean:19798.875 - actor/entropy:0.39622214436531067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6605160236358643 - training/rollout_probs_diff_mean:0.005419079214334488 - training/rollout_probs_diff_std:0.012383614666759968 - training/rollout_actor_probs_pearson_corr:0.998907744884491 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:0.9999876022338867 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1576137442025356e-05 - rollout_corr/rollout_is_max:1.7245110273361206 - rollout_corr/rollout_is_min:0.2930092215538025 - rollout_corr/rollout_is_std:0.03913719579577446 - rollout_corr/rollout_is_eff_sample_size:0.9984703847272242 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.002267190022394061 - rollout_corr/rollout_is_seq_max:1.0080516338348389 - rollout_corr/rollout_is_seq_min:0.9936971664428711 - rollout_corr/rollout_is_seq_max_deviation:0.008051633834838867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5080496412701905) - rollout_corr/training_log_ppl:np.float64(0.4042311883531511) - rollout_corr/kl:np.float64(0.0007036857559299392) - rollout_corr/k3_kl:np.float64(0.0009411578781737262) - rollout_corr/rollout_ppl:np.float64(1.5069640525616705) - rollout_corr/rollout_log_ppl:np.float64(0.40352750167949125) - rollout_corr/log_ppl_diff:np.float64(0.0007036866736598313) - rollout_corr/log_ppl_abs_diff:np.float64(0.001983315742108971) - rollout_corr/log_ppl_diff_max:np.float64(0.0076701343059539795) - rollout_corr/log_ppl_diff_min:np.float64(-0.006415665149688721) - rollout_corr/ppl_ratio:np.float64(1.0007067790720612) - rollout_corr/chi2_token:np.float64(0.002371277194470167) - rollout_corr/chi2_seq:np.float64(0.9433175600133836) - actor/pg_loss:np.float64(0.010519832605496049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009025154786286294) - actor/ppo_kl:np.float64(-0.00013643282175568316) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(12954.285722455475) - self_distillation/token_reweight_w_std:np.float64(227959.35449937586) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025726119056344) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14800434719654731) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.45579010248184204) - perf/mfu/actor:np.float64(0.04795526011784892) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.25069046020508) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013215383514761925 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013215383514761925 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:362.08984375 - response_length/max:864.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:362.08984375 - response_length_non_aborted/max:864.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.4523348808288574e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8027989882975817) - timing_s/agent_loop/generate_sequences/max:np.float64(7.439005896449089) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.104638532247918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.439005896449089) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:199 - timing_s/agent_loop/slowest/response_length:864 - timing_s/gen:14.218583673238754 - timing_s/reward:0.05999644473195076 - timing_s/old_log_prob:2.961333120241761 - timing_s/adv:0.5369714200496674 - timing_s/update_actor:21.455332016572356 - timing_s/step:39.31888073682785 - timing_s/stop_profile:0.00011654756963253021 - timing_per_token_ms/update_actor:0.1354580248661373 - timing_per_token_ms/gen:0.15339105316617677 - timing_per_token_ms/adv:0.00339016370911016 - perf/total_num_tokens:158391 - perf/time_per_step:39.31888073682785 - perf/throughput:503.54625129131597 (TaskRunner pid=2255397) Training Progress: 42%|████▏ | 42/100 [39:16<1:01:21, 63.48s/it] (TaskRunner pid=2255397) step:43 - global_seqlen/min:18065 - global_seqlen/max:23096 - global_seqlen/minmax_diff:5031 - global_seqlen/balanced_min:20756 - global_seqlen/balanced_max:20762 - global_seqlen/mean:20760.875 - actor/entropy:0.42775002121925354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424738645553589 - training/rollout_probs_diff_mean:0.005239918828010559 - training/rollout_probs_diff_std:0.01176416501402855 - training/rollout_actor_probs_pearson_corr:0.999097466468811 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:0.9999403357505798 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1294968317088205e-05 - rollout_corr/rollout_is_max:1.6902360916137695 - rollout_corr/rollout_is_min:0.39213547110557556 - rollout_corr/rollout_is_std:0.038980014622211456 - rollout_corr/rollout_is_eff_sample_size:0.9984827447477223 - rollout_corr/rollout_is_seq_mean:0.9998909831047058 - rollout_corr/rollout_is_seq_std:0.002513354877009988 - rollout_corr/rollout_is_seq_max:1.0098495483398438 - rollout_corr/rollout_is_seq_min:0.9915899038314819 - rollout_corr/rollout_is_seq_max_deviation:0.00984954833984375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5651745153591037) - rollout_corr/training_log_ppl:np.float64(0.4374735167075414) - rollout_corr/kl:np.float64(0.0007840867953743391) - rollout_corr/k3_kl:np.float64(0.0008964727625198066) - rollout_corr/rollout_ppl:np.float64(1.5639647683128715) - rollout_corr/rollout_log_ppl:np.float64(0.4366894279664848) - rollout_corr/log_ppl_diff:np.float64(0.0007840887410566211) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020498010562732816) - rollout_corr/log_ppl_diff_max:np.float64(0.009105801582336426) - rollout_corr/log_ppl_diff_min:np.float64(-0.00906282663345337) - rollout_corr/ppl_ratio:np.float64(1.0007877822499722) - rollout_corr/chi2_token:np.float64(0.00206923414953053) - rollout_corr/chi2_seq:np.float64(1.3589389722328633) - actor/pg_loss:np.float64(0.004318759543821216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004535638540801301) - actor/ppo_kl:np.float64(-0.00012510620860073374) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(39451.46745206439) - self_distillation/token_reweight_w_std:np.float64(461544.07824653096) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995274562388659) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09244409290840849) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.32581663131713867) - perf/mfu/actor:np.float64(0.05044233796581919) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.0129623413086) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011210255324840546 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011210255324840546 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.83984375 - response_length/max:801.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.83984375 - response_length_non_aborted/max:801.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001546517014503479 - timing_s/agent_loop/generate_sequences/min:np.float64(1.693193232640624) - timing_s/agent_loop/generate_sequences/max:np.float64(6.869511658325791) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.046683865577506) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.869511658325791) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:801 - timing_s/gen:14.301691135391593 - timing_s/reward:0.06055835448205471 - timing_s/old_log_prob:2.9482695516198874 - timing_s/adv:0.5146379265934229 - timing_s/update_actor:21.42802003212273 - timing_s/step:39.34179303236306 - timing_s/stop_profile:0.00010741874575614929 - timing_per_token_ms/update_actor:0.12901684076491676 - timing_per_token_ms/gen:0.16153714503181332 - timing_per_token_ms/adv:0.0030986045060325184 - perf/total_num_tokens:166087 - perf/time_per_step:39.34179303236306 - perf/throughput:527.7053585972008 (TaskRunner pid=2255397) Training Progress: 43%|████▎ | 43/100 [39:56<53:26, 56.25s/it] (TaskRunner pid=2255397) step:44 - global_seqlen/min:19213 - global_seqlen/max:22622 - global_seqlen/minmax_diff:3409 - global_seqlen/balanced_min:20879 - global_seqlen/balanced_max:20890 - global_seqlen/mean:20886.75 - actor/entropy:0.43929049372673035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22547024488449097 - training/rollout_probs_diff_mean:0.0053957682102918625 - training/rollout_probs_diff_std:0.011915630660951138 - training/rollout_actor_probs_pearson_corr:0.9990841150283813 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000036358833313 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9659661054611206 - rollout_corr/rollout_is_min:0.5844541192054749 - rollout_corr/rollout_is_std:0.039155468344688416 - rollout_corr/rollout_is_eff_sample_size:0.9984691962798411 - rollout_corr/rollout_is_seq_mean:1.0000959634780884 - rollout_corr/rollout_is_seq_std:0.0024589113891124725 - rollout_corr/rollout_is_seq_max:1.0087720155715942 - rollout_corr/rollout_is_seq_min:0.9921355843544006 - rollout_corr/rollout_is_seq_max_deviation:0.008772015571594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6047232481651008) - rollout_corr/training_log_ppl:np.float64(0.46260377258295193) - rollout_corr/kl:np.float64(0.0009372506496678756) - rollout_corr/k3_kl:np.float64(0.000976888309367041) - rollout_corr/rollout_ppl:np.float64(1.6032404340803623) - rollout_corr/rollout_log_ppl:np.float64(0.4616665196372196) - rollout_corr/log_ppl_diff:np.float64(0.0009372529457323253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002075349271763116) - rollout_corr/log_ppl_diff_max:np.float64(0.013183057308197021) - rollout_corr/log_ppl_diff_min:np.float64(-0.006263077259063721) - rollout_corr/ppl_ratio:np.float64(1.000940943369642) - rollout_corr/chi2_token:np.float64(0.0020958164241164923) - rollout_corr/chi2_seq:np.float64(0.34359696647152305) - actor/pg_loss:np.float64(0.005748242023400962) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005829084911965765) - actor/ppo_kl:np.float64(0.0002053698056716513) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(14803.191471951548) - self_distillation/token_reweight_w_std:np.float64(246306.37441055127) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9985546870157123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10855769729823805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.36420872062444687) - perf/mfu/actor:np.float64(0.05008791245123715) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.9407615661621) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00279244058765471 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00279244058765471 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.1484375 - response_length/max:1060.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.1484375 - response_length_non_aborted/max:1060.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014213845133781433 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7569324560463428) - timing_s/agent_loop/generate_sequences/max:np.float64(8.538507629185915) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.876661294561927) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.538507629185915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1060 - timing_s/gen:16.138035461306572 - timing_s/reward:0.06066634878516197 - timing_s/old_log_prob:3.082423932850361 - timing_s/adv:0.5534890294075012 - timing_s/update_actor:21.732121851295233 - timing_s/step:41.653873067349195 - timing_s/stop_profile:7.493607699871063e-05 - timing_per_token_ms/update_actor:0.13005925916726652 - timing_per_token_ms/gen:0.18478525501301407 - timing_per_token_ms/adv:0.00331244107752224 - perf/total_num_tokens:167094 - perf/time_per_step:41.653873067349195 - perf/throughput:501.4359640993934 (TaskRunner pid=2255397) Training Progress: 44%|████▍ | 44/100 [40:37<48:25, 51.89s/it] (TaskRunner pid=2255397) step:45 - global_seqlen/min:18835 - global_seqlen/max:26317 - global_seqlen/minmax_diff:7482 - global_seqlen/balanced_min:21440 - global_seqlen/balanced_max:21451 - global_seqlen/mean:21447.875 - actor/entropy:0.4101407527923584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28218671679496765 - training/rollout_probs_diff_mean:0.004841715097427368 - training/rollout_probs_diff_std:0.01116796862334013 - training/rollout_actor_probs_pearson_corr:0.9992083311080933 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9999707937240601 - rollout_corr/rollout_is_ratio_fraction_high:1.0986233974108472e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0986233974108472e-05 - rollout_corr/rollout_is_max:2.0019893646240234 - rollout_corr/rollout_is_min:0.25255540013313293 - rollout_corr/rollout_is_std:0.036969371140003204 - rollout_corr/rollout_is_eff_sample_size:0.998635012024894 - rollout_corr/rollout_is_seq_mean:1.0000436305999756 - rollout_corr/rollout_is_seq_std:0.002479569287970662 - rollout_corr/rollout_is_seq_max:1.0116991996765137 - rollout_corr/rollout_is_seq_min:0.9924298524856567 - rollout_corr/rollout_is_seq_max_deviation:0.011699199676513672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5578296538442373) - rollout_corr/training_log_ppl:np.float64(0.432097623241134) - rollout_corr/kl:np.float64(0.0007768278248150295) - rollout_corr/k3_kl:np.float64(0.0008250089271086836) - rollout_corr/rollout_ppl:np.float64(1.5565427541732788) - rollout_corr/rollout_log_ppl:np.float64(0.4313207929953933) - rollout_corr/log_ppl_diff:np.float64(0.0007768302457407117) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019471754785627127) - rollout_corr/log_ppl_diff_max:np.float64(0.011698126792907715) - rollout_corr/log_ppl_diff_min:np.float64(-0.010318875312805176) - rollout_corr/ppl_ratio:np.float64(1.0007803542539477) - rollout_corr/chi2_token:np.float64(0.0017588012851774693) - rollout_corr/chi2_seq:np.float64(0.6141417354810983) - actor/pg_loss:np.float64(0.0026712483959272504) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039884855823402177) - actor/ppo_kl:np.float64(6.892802709934642e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(38907.24203802785) - self_distillation/token_reweight_w_std:np.float64(568287.8935087718) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006591160781682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08028327141073532) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.28873957321047783) - perf/mfu/actor:np.float64(0.05157141247510403) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.14898681640625) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005405227188020945 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005405227188020945 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.55859375 - response_length/max:782.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.55859375 - response_length_non_aborted/max:782.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7983601689338684e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.374290145933628) - timing_s/agent_loop/generate_sequences/max:np.float64(7.021162023767829) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9449067628447665) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.021162023767829) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:13.897942123934627 - timing_s/reward:0.06329947151243687 - timing_s/old_log_prob:3.025046695023775 - timing_s/adv:0.56037213280797 - timing_s/update_actor:21.67712887376547 - timing_s/step:39.31348724476993 - timing_s/stop_profile:0.00011695176362991333 - timing_per_token_ms/update_actor:0.1263361106506208 - timing_per_token_ms/gen:0.15268604774545583 - timing_per_token_ms/adv:0.0032658954139277788 - perf/total_num_tokens:171583 - perf/time_per_step:39.31348724476993 - perf/throughput:545.5602263534461 (TaskRunner pid=2255397) Training Progress: 45%|████▌ | 45/100 [41:17<44:07, 48.13s/it] (TaskRunner pid=2255397) step:46 - global_seqlen/min:17315 - global_seqlen/max:21947 - global_seqlen/minmax_diff:4632 - global_seqlen/balanced_min:19767 - global_seqlen/balanced_max:19776 - global_seqlen/mean:19773.25 - actor/entropy:0.4300471842288971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5463374257087708 - training/rollout_probs_diff_mean:0.0050852419808506966 - training/rollout_probs_diff_std:0.011606025509536266 - training/rollout_actor_probs_pearson_corr:0.9991208910942078 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8203125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8203125 - rollout_corr/rollout_is_mean:0.9999197125434875 - rollout_corr/rollout_is_ratio_fraction_high:1.1397570233384613e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2795140466769226e-05 - rollout_corr/rollout_is_max:2.4014859199523926 - rollout_corr/rollout_is_min:0.26859816908836365 - rollout_corr/rollout_is_std:0.03753577917814255 - rollout_corr/rollout_is_eff_sample_size:0.9985928099624436 - rollout_corr/rollout_is_seq_mean:1.0000156164169312 - rollout_corr/rollout_is_seq_std:0.002315930090844631 - rollout_corr/rollout_is_seq_max:1.0097026824951172 - rollout_corr/rollout_is_seq_min:0.9926007986068726 - rollout_corr/rollout_is_seq_max_deviation:0.009702682495117188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5882981927134097) - rollout_corr/training_log_ppl:np.float64(0.44962009048322216) - rollout_corr/kl:np.float64(0.0011486389046666545) - rollout_corr/k3_kl:np.float64(0.0009466409361493788) - rollout_corr/rollout_ppl:np.float64(1.5864842752926052) - rollout_corr/rollout_log_ppl:np.float64(0.4484714485006407) - rollout_corr/log_ppl_diff:np.float64(0.0011486419825814664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022174433688633144) - rollout_corr/log_ppl_diff_max:np.float64(0.015058279037475586) - rollout_corr/log_ppl_diff_min:np.float64(-0.00858527421951294) - rollout_corr/ppl_ratio:np.float64(1.0011532127391547) - rollout_corr/chi2_token:np.float64(0.0015598325990140438) - rollout_corr/chi2_seq:np.float64(0.6058120087254792) - actor/pg_loss:np.float64(0.005358678288757801) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006527249729515461) - actor/ppo_kl:np.float64(0.0004008172310321356) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(13853.629367629066) - self_distillation/token_reweight_w_std:np.float64(223926.18253548152) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000649391906336) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12465728094684891) - self_distillation/empty_target_batch:np.float64(0.1796875) - actor/grad_norm:np.float64(0.37411294877529144) - perf/mfu/actor:np.float64(0.04759140321566382) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.96075439453125) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004386639688163996 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004386639688163996 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:342.7265625 - response_length/max:874.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:342.7265625 - response_length_non_aborted/max:874.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012195669114589691 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6501337420195341) - timing_s/agent_loop/generate_sequences/max:np.float64(7.341919541358948) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8824371568043716) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.341919541358948) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:177 - timing_s/agent_loop/slowest/response_length:874 - timing_s/gen:14.224360808730125 - timing_s/reward:0.07079944014549255 - timing_s/old_log_prob:2.932241167873144 - timing_s/adv:0.537244874984026 - timing_s/update_actor:21.595252191647887 - timing_s/step:39.45111795514822 - timing_s/stop_profile:0.00012188218533992767 - timing_per_token_ms/update_actor:0.13651810015834454 - timing_per_token_ms/gen:0.16212314856424953 - timing_per_token_ms/adv:0.0033962858595831865 - perf/total_num_tokens:158186 - perf/time_per_step:39.45111795514822 - perf/throughput:501.2088636494436 (TaskRunner pid=2255397) Training Progress: 46%|████▌ | 46/100 [41:56<40:59, 45.54s/it] (TaskRunner pid=2255397) step:47 - global_seqlen/min:16338 - global_seqlen/max:23481 - global_seqlen/minmax_diff:7143 - global_seqlen/balanced_min:20960 - global_seqlen/balanced_max:20969 - global_seqlen/mean:20966.0 - actor/entropy:0.4562165141105652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135512113571167 - training/rollout_probs_diff_mean:0.004941638559103012 - training/rollout_probs_diff_std:0.01093968003988266 - training/rollout_actor_probs_pearson_corr:0.9992673993110657 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0000667572021484 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9514927864074707 - rollout_corr/rollout_is_min:0.5686209797859192 - rollout_corr/rollout_is_std:0.03698226809501648 - rollout_corr/rollout_is_eff_sample_size:0.998634417604882 - rollout_corr/rollout_is_seq_mean:1.0001049041748047 - rollout_corr/rollout_is_seq_std:0.0021275379694998264 - rollout_corr/rollout_is_seq_max:1.0070573091506958 - rollout_corr/rollout_is_seq_min:0.9932472705841064 - rollout_corr/rollout_is_seq_max_deviation:0.007057309150695801 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6164640178903937) - rollout_corr/training_log_ppl:np.float64(0.46599303945549764) - rollout_corr/kl:np.float64(0.0011402188001117253) - rollout_corr/k3_kl:np.float64(0.0008741923521711215) - rollout_corr/rollout_ppl:np.float64(1.6145951971411705) - rollout_corr/rollout_log_ppl:np.float64(0.46485281962668523) - rollout_corr/log_ppl_diff:np.float64(0.0011402198288124055) - rollout_corr/log_ppl_abs_diff:np.float64(0.001965065923286602) - rollout_corr/log_ppl_diff_max:np.float64(0.008194833993911743) - rollout_corr/log_ppl_diff_min:np.float64(-0.005479037761688232) - rollout_corr/ppl_ratio:np.float64(1.0011436399072409) - rollout_corr/chi2_token:np.float64(0.0012568566016852856) - rollout_corr/chi2_seq:np.float64(0.40265263244509697) - actor/pg_loss:np.float64(0.006021094392053783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619370990643802) - actor/ppo_kl:np.float64(0.0005166811963022155) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(12667.028565393994) - self_distillation/token_reweight_w_std:np.float64(223391.28332038334) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007588975131512) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12597810916486196) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.35431264340877533) - perf/mfu/actor:np.float64(0.050087304585172576) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.9956817626953) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005755655933171511 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005755655933171511 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.90625 - response_length/max:784.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.90625 - response_length_non_aborted/max:784.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.334273636341095e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7290426678955555) - timing_s/agent_loop/generate_sequences/max:np.float64(7.266604153439403) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.170146645323257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.266604153439403) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:784 - timing_s/gen:14.346795970574021 - timing_s/reward:0.06195123679935932 - timing_s/old_log_prob:2.8864920753985643 - timing_s/adv:0.5422863606363535 - timing_s/update_actor:21.55750539712608 - timing_s/step:39.47956304065883 - timing_s/stop_profile:2.954714000225067e-05 - timing_per_token_ms/update_actor:0.12852657515218735 - timing_per_token_ms/gen:0.1548527326070074 - timing_per_token_ms/adv:0.0032331295945599633 - perf/total_num_tokens:167728 - perf/time_per_step:39.47956304065883 - perf/throughput:531.0595757710828 (TaskRunner pid=2255397) Training Progress: 47%|████▋ | 47/100 [42:36<38:37, 43.73s/it] (TaskRunner pid=2255397) step:48 - global_seqlen/min:16138 - global_seqlen/max:20539 - global_seqlen/minmax_diff:4401 - global_seqlen/balanced_min:19360 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19365.625 - actor/entropy:0.45748016238212585 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.423759400844574 - training/rollout_probs_diff_mean:0.00517674908041954 - training/rollout_probs_diff_std:0.011389809660613537 - training/rollout_actor_probs_pearson_corr:0.9992020130157471 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000624656677246 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.4522043279139325e-05 - rollout_corr/rollout_is_max:1.9825339317321777 - rollout_corr/rollout_is_min:0.29720374941825867 - rollout_corr/rollout_is_std:0.038425594568252563 - rollout_corr/rollout_is_eff_sample_size:0.9985258883712781 - rollout_corr/rollout_is_seq_mean:1.0000547170639038 - rollout_corr/rollout_is_seq_std:0.0022232853807508945 - rollout_corr/rollout_is_seq_max:1.0069233179092407 - rollout_corr/rollout_is_seq_min:0.9934024810791016 - rollout_corr/rollout_is_seq_max_deviation:0.006923317909240723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6189399659633636) - rollout_corr/training_log_ppl:np.float64(0.47103061771485955) - rollout_corr/kl:np.float64(0.0008533685957203829) - rollout_corr/k3_kl:np.float64(0.0008472814761262271) - rollout_corr/rollout_ppl:np.float64(1.6175275961868465) - rollout_corr/rollout_log_ppl:np.float64(0.47017724718898535) - rollout_corr/log_ppl_diff:np.float64(0.0008533705258741975) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018438339466229081) - rollout_corr/log_ppl_diff_max:np.float64(0.009492874145507812) - rollout_corr/log_ppl_diff_min:np.float64(-0.004758328199386597) - rollout_corr/ppl_ratio:np.float64(1.000856165541336) - rollout_corr/chi2_token:np.float64(0.0016905255615711212) - rollout_corr/chi2_seq:np.float64(0.5319139340426773) - actor/pg_loss:np.float64(0.004102788749150932) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002375331782786816) - actor/ppo_kl:np.float64(0.00012016280814464153) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(1381.5336034777574) - self_distillation/token_reweight_w_std:np.float64(20333.09212728584) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9975523005705327) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11302358307875693) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.2960100993514061) - perf/mfu/actor:np.float64(0.04606089601890587) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.1104965209961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013794433325529099 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013794433325529099 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.45703125 - response_length/max:832.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.45703125 - response_length_non_aborted/max:832.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.479366958141327e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9788021426647902) - timing_s/agent_loop/generate_sequences/max:np.float64(7.109138231724501) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9673785015329486) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.109138231724501) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:832 - timing_s/gen:13.836422380059958 - timing_s/reward:0.06020788662135601 - timing_s/old_log_prob:2.906719511374831 - timing_s/adv:0.5168796796351671 - timing_s/update_actor:21.836700728163123 - timing_s/step:39.24484575167298 - timing_s/stop_profile:0.00012063421308994293 - timing_per_token_ms/update_actor:0.1409501418632443 - timing_per_token_ms/gen:0.1592205196725004 - timing_per_token_ms/adv:0.0033363219598848935 - perf/total_num_tokens:154925 - perf/time_per_step:39.24484575167298 - perf/throughput:493.4565196800259 (TaskRunner pid=2255397) Training Progress: 48%|████▊ | 48/100 [43:15<36:44, 42.40s/it] (TaskRunner pid=2255397) step:49 - global_seqlen/min:19946 - global_seqlen/max:22692 - global_seqlen/minmax_diff:2746 - global_seqlen/balanced_min:21730 - global_seqlen/balanced_max:21733 - global_seqlen/mean:21731.75 - actor/entropy:0.4979252517223358 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43991324305534363 - training/rollout_probs_diff_mean:0.004984842147678137 - training/rollout_probs_diff_std:0.01066806074231863 - training/rollout_actor_probs_pearson_corr:0.9993179440498352 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998939037322998 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.758192391018383e-06 - rollout_corr/rollout_is_max:1.9085274934768677 - rollout_corr/rollout_is_min:0.12017349898815155 - rollout_corr/rollout_is_std:0.03726325184106827 - rollout_corr/rollout_is_eff_sample_size:0.9986130189557251 - rollout_corr/rollout_is_seq_mean:0.9998945593833923 - rollout_corr/rollout_is_seq_std:0.001821837155148387 - rollout_corr/rollout_is_seq_max:1.005088448524475 - rollout_corr/rollout_is_seq_min:0.9940054416656494 - rollout_corr/rollout_is_seq_max_deviation:0.005994558334350586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6754766842350364) - rollout_corr/training_log_ppl:np.float64(0.5048167807399295) - rollout_corr/kl:np.float64(0.0009508746554258707) - rollout_corr/k3_kl:np.float64(0.0007735003869697721) - rollout_corr/rollout_ppl:np.float64(1.6738910162821412) - rollout_corr/rollout_log_ppl:np.float64(0.503865905397106) - rollout_corr/log_ppl_diff:np.float64(0.000950875342823565) - rollout_corr/log_ppl_abs_diff:np.float64(0.001617251429706812) - rollout_corr/log_ppl_diff_max:np.float64(0.008522868156433105) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493285059928894) - rollout_corr/ppl_ratio:np.float64(1.0009531062096357) - rollout_corr/chi2_token:np.float64(0.0011934449430555105) - rollout_corr/chi2_seq:np.float64(0.1550909043289721) - actor/pg_loss:np.float64(0.005771090276539326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002795115901790268) - actor/ppo_kl:np.float64(0.00010383528175317291) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(4803.391529331682) - self_distillation/token_reweight_w_std:np.float64(100370.09618178767) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001389476004988) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10803735570516437) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3376929610967636) - perf/mfu/actor:np.float64(0.05224311765341433) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.75490951538086) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0023309881798923016 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0023309881798923016 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:400.3046875 - response_length/max:826.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:400.3046875 - response_length_non_aborted/max:826.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013683363795280457 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6328133698552847) - timing_s/agent_loop/generate_sequences/max:np.float64(7.4781682174652815) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.834140434708388) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.4781682174652815) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:826 - timing_s/gen:14.471364917233586 - timing_s/reward:0.060647664591670036 - timing_s/old_log_prob:2.9566055797040462 - timing_s/adv:0.5396076701581478 - timing_s/update_actor:21.56921719573438 - timing_s/step:39.68868784233928 - timing_s/stop_profile:2.8738752007484436e-05 - timing_per_token_ms/update_actor:0.12406511898336754 - timing_per_token_ms/gen:0.14121435739606147 - timing_per_token_ms/adv:0.0031037978427769725 - perf/total_num_tokens:173854 - perf/time_per_step:39.68868784233928 - perf/throughput:547.5552652767952 (TaskRunner pid=2255397) Training Progress: 49%|████▉ | 49/100 [43:55<35:21, 41.59s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to examine the structure of the product and compare it with the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has a nitrogen-containing ring with a substituent group: "Cc2ccc(Cl)cc2C(F)(F)F". This suggests that the molecule contains a chlorinated aromatic ring attached to a trifluoromethyl group, which is connected to a nitrogen-containing ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's evaluate the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option A**: Contains a structure with a nitrogen ring and a chlorinated aromatic ring connected to a trifluoromethyl group. The structure is very similar to the product, with the correct substituents and connectivity. (TaskRunner pid=2255397) - **Option B**: Contains a chlorinated nitrogen ring but lacks the trifluoromethyl group and the correct aromatic ring structure. (TaskRunner pid=2255397) - **Option C**: Contains a hydroxyl group and a different substituent, not matching the product. (TaskRunner pid=2255397) - **Option D**: Contains a different ring structure and lacks the trifluoromethyl group and the correct aromatic ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Only **Option A** matches the structure and substituents of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2255397) step:50 - global_seqlen/min:19045 - global_seqlen/max:30156 - global_seqlen/minmax_diff:11111 - global_seqlen/balanced_min:21698 - global_seqlen/balanced_max:21699 - global_seqlen/mean:21698.5 - actor/entropy:0.39573362469673157 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491804838180542 - training/rollout_probs_diff_mean:0.004589053802192211 - training/rollout_probs_diff_std:0.01072707585990429 - training/rollout_actor_probs_pearson_corr:0.9992213845252991 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000442266464233 - rollout_corr/rollout_is_ratio_fraction_high:9.806613888940774e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.004254102706909 - rollout_corr/rollout_is_min:0.6065316796302795 - rollout_corr/rollout_is_std:0.03603039309382439 - rollout_corr/rollout_is_eff_sample_size:0.9987034939469002 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0021530245430767536 - rollout_corr/rollout_is_seq_max:1.0064139366149902 - rollout_corr/rollout_is_seq_min:0.991096556186676 - rollout_corr/rollout_is_seq_max_deviation:0.008903443813323975 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.524656287394464) - rollout_corr/training_log_ppl:np.float64(0.4108033778902609) - rollout_corr/kl:np.float64(0.0007370334822254243) - rollout_corr/k3_kl:np.float64(0.0008599899813646061) - rollout_corr/rollout_ppl:np.float64(1.5234609101898968) - rollout_corr/rollout_log_ppl:np.float64(0.41006634262157604) - rollout_corr/log_ppl_diff:np.float64(0.0007370352686848491) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018268449639435858) - rollout_corr/log_ppl_diff_max:np.float64(0.00987979769706726) - rollout_corr/log_ppl_diff_min:np.float64(-0.00624692440032959) - rollout_corr/ppl_ratio:np.float64(1.000740017509088) - rollout_corr/chi2_token:np.float64(0.0019624873530119658) - rollout_corr/chi2_seq:np.float64(0.6737326441798359) - actor/pg_loss:np.float64(0.00585430464707315) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007581320141980541) - actor/ppo_kl:np.float64(0.00011956099752552518) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(4577.494679181604) - self_distillation/token_reweight_w_std:np.float64(75540.59034832189) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9987221618648618) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11705390218412504) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3779703937470913) - perf/mfu/actor:np.float64(0.0522593230068327) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.6965789794922) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.66875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17551919057486814) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372904761904763) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13660825245257055) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6003428571428571) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15818906699634022) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6687047619047619) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17558003827147262) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7877285714285714) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09825969433764362) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5388809523809523) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13247872360110166) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6810333333333333) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13288868774672) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8258047619047618) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07020796516216546) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4855857142857143) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10698787291527515) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6865809523809525) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09029984972979069) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8547666666666667) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04666779709351366) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4396285714285714) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07582314271837051) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6883333333333334) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06437108221973292) - val-aux/sciknoweval/score/mean@16:np.float64(0.66875) - val-aux/sciknoweval/score/std@16:np.float64(0.17551919057486814) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7372904761904763) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13660825245257055) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6003428571428571) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15818906699634022) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6687047619047619) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17558003827147262) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7877285714285714) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09825969433764362) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5388809523809523) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13247872360110166) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6810333333333333) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13288868774672) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8258047619047618) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07020796516216546) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4855857142857143) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10698787291527515) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6865809523809525) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09029984972979069) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8547666666666667) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04666779709351366) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4396285714285714) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07582314271837051) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6883333333333334) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06437108221973292) - val-core/sciknoweval/acc/mean@16:np.float64(0.66875) - val-aux/sciknoweval/acc/std@16:np.float64(0.17551919057486814) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7372904761904763) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13660825245257055) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6003428571428571) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15818906699634022) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6687047619047619) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17558003827147262) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7877285714285714) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09825969433764362) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5388809523809523) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13247872360110166) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6810333333333333) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13288868774672) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8258047619047618) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07020796516216546) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4855857142857143) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10698787291527515) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6865809523809525) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09029984972979069) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8547666666666667) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04666779709351366) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4396285714285714) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07582314271837051) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6883333333333334) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06437108221973292) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9985119047619048) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.005763368074713418) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998666666666667) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.001758076041122951) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9971714285714286) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.007695791728287569) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9986190476190475) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.00554804164831311) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9947428571428573) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.00986918416859868) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9995904761904763) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.003032413337024054) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9906380952380953) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.011629673038340291) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999619047619047) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0006991297950679286) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9847428571428571) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.011415170996457474) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00015050933932646775) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015592515701428056 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015592515701428056 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.328125 - response_length/max:872.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.328125 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.404837429523468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0187409687787294) - timing_s/agent_loop/generate_sequences/max:np.float64(7.631963180378079) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.57371993160632) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.631963180378079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:872 - timing_s/gen:14.39490077085793 - timing_s/reward:0.06266830302774906 - timing_s/old_log_prob:3.103602586314082 - timing_s/adv:0.5463018678128719 - timing_s/update_actor:21.542298316955566 - timing_s/step:39.73873019963503 - timing_s/testing:133.93844243697822 - timing_s/save_checkpoint:13.662320464849472 - timing_s/stop_profile:0.0001660715788602829 - timing_per_token_ms/update_actor:0.12410015851876607 - timing_per_token_ms/gen:0.14116522938510503 - timing_per_token_ms/adv:0.003147117702910754 - perf/total_num_tokens:173588 - perf/time_per_step:39.73873019963503 - perf/throughput:546.0290223415162 (TaskRunner pid=2255397) Training Progress: 50%|█████ | 50/100 [47:02<1:11:07, 85.35s/it] (TaskRunner pid=2255397) step:51 - global_seqlen/min:19988 - global_seqlen/max:24236 - global_seqlen/minmax_diff:4248 - global_seqlen/balanced_min:21993 - global_seqlen/balanced_max:22000 - global_seqlen/mean:21997.625 - actor/entropy:0.47643905878067017 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292527914047241 - training/rollout_probs_diff_mean:0.004974652547389269 - training/rollout_probs_diff_std:0.010949385352432728 - training/rollout_actor_probs_pearson_corr:0.9992679953575134 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000044584274292 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0419596947031096e-05 - rollout_corr/rollout_is_max:1.6338132619857788 - rollout_corr/rollout_is_min:0.19046327471733093 - rollout_corr/rollout_is_std:0.03751512989401817 - rollout_corr/rollout_is_eff_sample_size:0.998594711950472 - rollout_corr/rollout_is_seq_mean:0.9999472498893738 - rollout_corr/rollout_is_seq_std:0.002286979230120778 - rollout_corr/rollout_is_seq_max:1.0055614709854126 - rollout_corr/rollout_is_seq_min:0.9931483864784241 - rollout_corr/rollout_is_seq_max_deviation:0.006851613521575928 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6695182812400162) - rollout_corr/training_log_ppl:np.float64(0.49964700039708987) - rollout_corr/kl:np.float64(0.0010526214147859037) - rollout_corr/k3_kl:np.float64(0.0008514240415706809) - rollout_corr/rollout_ppl:np.float64(1.6677322825416923) - rollout_corr/rollout_log_ppl:np.float64(0.49859437631675974) - rollout_corr/log_ppl_diff:np.float64(0.0010526240803301334) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019851572578772902) - rollout_corr/log_ppl_diff_max:np.float64(0.008082211017608643) - rollout_corr/log_ppl_diff_min:np.float64(-0.008058786392211914) - rollout_corr/ppl_ratio:np.float64(1.0010559679940343) - rollout_corr/chi2_token:np.float64(0.0013342886231839657) - rollout_corr/chi2_seq:np.float64(1.1409833203069866) - actor/pg_loss:np.float64(0.005735161481425166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006951400860089052) - actor/ppo_kl:np.float64(0.00024209431599508946) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(7385.579081017524) - self_distillation/token_reweight_w_std:np.float64(136267.17127367936) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020049593877047) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11803711034008302) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3889169320464134) - perf/mfu/actor:np.float64(0.053154140926217126) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.41073989868164) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006512248073704541 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006512248073704541 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.89453125 - response_length/max:928.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.89453125 - response_length_non_aborted/max:928.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013144873082637787 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7525868192315102) - timing_s/agent_loop/generate_sequences/max:np.float64(7.869243094697595) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.208094254077878) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.869243094697595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:928 - timing_s/gen:14.724404215812683 - timing_s/reward:0.059697989374399185 - timing_s/old_log_prob:3.061056010425091 - timing_s/adv:0.5633108653128147 - timing_s/update_actor:21.545322271063924 - timing_s/step:40.06024991162121 - timing_s/stop_profile:0.00010966137051582336 - timing_per_token_ms/update_actor:0.12242982066850355 - timing_per_token_ms/gen:0.15342236062030656 - timing_per_token_ms/adv:0.00320097547640265 - perf/total_num_tokens:175981 - perf/time_per_step:40.06024991162121 - perf/throughput:549.1135239677732 (TaskRunner pid=2255397) Training Progress: 51%|█████ | 51/100 [47:42<58:37, 71.78s/it] (TaskRunner pid=2255397) step:52 - global_seqlen/min:16418 - global_seqlen/max:23760 - global_seqlen/minmax_diff:7342 - global_seqlen/balanced_min:20391 - global_seqlen/balanced_max:20400 - global_seqlen/mean:20398.0 - actor/entropy:0.4327685832977295 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141288995742798 - training/rollout_probs_diff_mean:0.004852033220231533 - training/rollout_probs_diff_std:0.011078495532274246 - training/rollout_actor_probs_pearson_corr:0.9992164969444275 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.999881386756897 - rollout_corr/rollout_is_ratio_fraction_high:1.0429704161651898e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.107421636581421 - rollout_corr/rollout_is_min:0.5139061212539673 - rollout_corr/rollout_is_std:0.03718959912657738 - rollout_corr/rollout_is_eff_sample_size:0.9986184874121644 - rollout_corr/rollout_is_seq_mean:0.9998266696929932 - rollout_corr/rollout_is_seq_std:0.0023117384407669306 - rollout_corr/rollout_is_seq_max:1.010014533996582 - rollout_corr/rollout_is_seq_min:0.9893986582756042 - rollout_corr/rollout_is_seq_max_deviation:0.010601341724395752 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5747977024875581) - rollout_corr/training_log_ppl:np.float64(0.44207731005735695) - rollout_corr/kl:np.float64(0.0007742779266513011) - rollout_corr/k3_kl:np.float64(0.0008074032141394127) - rollout_corr/rollout_ppl:np.float64(1.5735519244335592) - rollout_corr/rollout_log_ppl:np.float64(0.44130302700796165) - rollout_corr/log_ppl_diff:np.float64(0.0007742830493953079) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017560763808432966) - rollout_corr/log_ppl_diff_max:np.float64(0.011528313159942627) - rollout_corr/log_ppl_diff_min:np.float64(-0.008214741945266724) - rollout_corr/ppl_ratio:np.float64(1.0007772489916533) - rollout_corr/chi2_token:np.float64(0.0016987936105579138) - rollout_corr/chi2_seq:np.float64(0.6159643346909434) - actor/pg_loss:np.float64(0.004326079739257693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003785648150369525) - actor/ppo_kl:np.float64(-0.00013583274305517534) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(23087.692809279542) - self_distillation/token_reweight_w_std:np.float64(411971.67904203804) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013474761508405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10722107632318512) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.31624583154916763) - perf/mfu/actor:np.float64(0.04994189212775807) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.2727928161621) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00683667091652751 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00683667091652751 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.53125 - response_length/max:748.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.53125 - response_length_non_aborted/max:748.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.608851253986359e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5058961473405361) - timing_s/agent_loop/generate_sequences/max:np.float64(6.993606846779585) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.290860692432034) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.993606846779585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:748 - timing_s/gen:13.7207720130682 - timing_s/reward:0.060612039640545845 - timing_s/old_log_prob:2.9474493078887463 - timing_s/adv:0.5020784568041563 - timing_s/update_actor:21.135599818080664 - timing_s/step:38.45533044077456 - timing_s/stop_profile:0.0001184120774269104 - timing_per_token_ms/update_actor:0.1295200498705796 - timing_per_token_ms/gen:0.14310358795440342 - timing_per_token_ms/adv:0.0030767627757877997 - perf/total_num_tokens:163184 - perf/time_per_step:38.45533044077456 - perf/throughput:530.4336165155352 (TaskRunner pid=2255397) Training Progress: 52%|█████▏ | 52/100 [48:21<49:26, 61.79s/it] (TaskRunner pid=2255397) step:53 - global_seqlen/min:19241 - global_seqlen/max:23446 - global_seqlen/minmax_diff:4205 - global_seqlen/balanced_min:21257 - global_seqlen/balanced_max:21274 - global_seqlen/mean:21261.625 - actor/entropy:0.4501931071281433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294929027557373 - training/rollout_probs_diff_mean:0.0048794192261993885 - training/rollout_probs_diff_std:0.010888302698731422 - training/rollout_actor_probs_pearson_corr:0.9992465376853943 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998956918716431 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1179934265092015e-05 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.37676042318344116 - rollout_corr/rollout_is_std:0.036510247737169266 - rollout_corr/rollout_is_eff_sample_size:0.9986683006749928 - rollout_corr/rollout_is_seq_mean:0.9998424053192139 - rollout_corr/rollout_is_seq_std:0.0020514947827905416 - rollout_corr/rollout_is_seq_max:1.0057796239852905 - rollout_corr/rollout_is_seq_min:0.9921916127204895 - rollout_corr/rollout_is_seq_max_deviation:0.007808387279510498 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6163378930650651) - rollout_corr/training_log_ppl:np.float64(0.4686294782150071) - rollout_corr/kl:np.float64(0.000762243592463463) - rollout_corr/k3_kl:np.float64(0.000810159646334796) - rollout_corr/rollout_ppl:np.float64(1.6150716617703438) - rollout_corr/rollout_log_ppl:np.float64(0.46786723379045725) - rollout_corr/log_ppl_diff:np.float64(0.0007622444245498627) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019291590724606067) - rollout_corr/log_ppl_diff_max:np.float64(0.008153051137924194) - rollout_corr/log_ppl_diff_min:np.float64(-0.005449235439300537) - rollout_corr/ppl_ratio:np.float64(1.000765364151448) - rollout_corr/chi2_token:np.float64(0.001715675462037325) - rollout_corr/chi2_seq:np.float64(0.6910448384005576) - actor/pg_loss:np.float64(0.0023863104870542884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002726298084780865) - actor/ppo_kl:np.float64(-0.00011610987491650349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(1507.4308137190528) - self_distillation/token_reweight_w_std:np.float64(33027.9989340055) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995625410228968) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06903543486259878) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.21907289698719978) - perf/mfu/actor:np.float64(0.051939957351732395) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.24185180664062) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003284213598817587 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003284213598817587 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:368.86328125 - response_length/max:1113.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:368.86328125 - response_length_non_aborted/max:1113.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.354894816875458e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2522868532687426) - timing_s/agent_loop/generate_sequences/max:np.float64(8.869224239140749) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.229014496719174) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.869224239140749) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:227 - timing_s/agent_loop/slowest/response_length:1113 - timing_s/gen:15.673873921856284 - timing_s/reward:0.061943380162119865 - timing_s/old_log_prob:2.946569699794054 - timing_s/adv:0.5094821080565453 - timing_s/update_actor:21.332264576107264 - timing_s/step:40.6122511010617 - timing_s/stop_profile:0.00011801905930042267 - timing_per_token_ms/update_actor:0.1254152997248991 - timing_per_token_ms/gen:0.16598580861659326 - timing_per_token_ms/adv:0.0029953149633232716 - perf/total_num_tokens:170093 - perf/time_per_step:40.6122511010617 - perf/throughput:523.5273697853249 (TaskRunner pid=2255397) Training Progress: 53%|█████▎ | 53/100 [49:01<43:26, 55.45s/it] (TaskRunner pid=2255397) step:54 - global_seqlen/min:16771 - global_seqlen/max:23434 - global_seqlen/minmax_diff:6663 - global_seqlen/balanced_min:20121 - global_seqlen/balanced_max:20134 - global_seqlen/mean:20129.375 - actor/entropy:0.39255428314208984 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2144627422094345 - training/rollout_probs_diff_mean:0.004672019276767969 - training/rollout_probs_diff_std:0.011115854606032372 - training/rollout_actor_probs_pearson_corr:0.9991840720176697 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999427795410156 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.969622254371643 - rollout_corr/rollout_is_min:0.5503029227256775 - rollout_corr/rollout_is_std:0.03688544034957886 - rollout_corr/rollout_is_eff_sample_size:0.998641194034968 - rollout_corr/rollout_is_seq_mean:0.9999832510948181 - rollout_corr/rollout_is_seq_std:0.002377620432525873 - rollout_corr/rollout_is_seq_max:1.0109039545059204 - rollout_corr/rollout_is_seq_min:0.9921522736549377 - rollout_corr/rollout_is_seq_max_deviation:0.01090395450592041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5304169207811356) - rollout_corr/training_log_ppl:np.float64(0.4121860411542002) - rollout_corr/kl:np.float64(0.0008754907883350427) - rollout_corr/k3_kl:np.float64(0.0008680712330715323) - rollout_corr/rollout_ppl:np.float64(1.5290372776798904) - rollout_corr/rollout_log_ppl:np.float64(0.411310548952315) - rollout_corr/log_ppl_diff:np.float64(0.0008754922018852085) - rollout_corr/log_ppl_abs_diff:np.float64(0.002070396934868768) - rollout_corr/log_ppl_diff_max:np.float64(0.016439110040664673) - rollout_corr/log_ppl_diff_min:np.float64(-0.00867973268032074) - rollout_corr/ppl_ratio:np.float64(1.0008794867899269) - rollout_corr/chi2_token:np.float64(0.0017218261491507292) - rollout_corr/chi2_seq:np.float64(0.5381895105820149) - actor/pg_loss:np.float64(0.006413624971173704) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005543413315081125) - actor/ppo_kl:np.float64(9.862682850325655e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(21560.225262402324) - self_distillation/token_reweight_w_std:np.float64(340430.1742267248) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004913297016174) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10845586232608184) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3543579652905464) - perf/mfu/actor:np.float64(0.04868076222396018) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.25546646118164) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011497573927044868 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011497573927044868 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:358.26171875 - response_length/max:1148.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:358.26171875 - response_length_non_aborted/max:1148.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010633282363414764 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4096278883516788) - timing_s/agent_loop/generate_sequences/max:np.float64(9.100342744961381) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.021014133017161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.100342744961381) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:1148 - timing_s/gen:15.916911805048585 - timing_s/reward:0.059211257845163345 - timing_s/old_log_prob:3.0376540310680866 - timing_s/adv:0.5192954838275909 - timing_s/update_actor:21.510440975427628 - timing_s/step:41.131219344213605 - timing_s/stop_profile:3.115646541118622e-05 - timing_per_token_ms/update_actor:0.13357618514874176 - timing_per_token_ms/gen:0.17354753099327902 - timing_per_token_ms/adv:0.003224736758019008 - perf/total_num_tokens:161035 - perf/time_per_step:41.131219344213605 - perf/throughput:489.3940739160661 (TaskRunner pid=2255397) Training Progress: 54%|█████▍ | 54/100 [49:42<39:13, 51.16s/it] (TaskRunner pid=2255397) step:55 - global_seqlen/min:19138 - global_seqlen/max:21905 - global_seqlen/minmax_diff:2767 - global_seqlen/balanced_min:20379 - global_seqlen/balanced_max:20392 - global_seqlen/mean:20387.375 - actor/entropy:0.46607786417007446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44003328680992126 - training/rollout_probs_diff_mean:0.005262596998363733 - training/rollout_probs_diff_std:0.011458019725978374 - training/rollout_actor_probs_pearson_corr:0.9992002844810486 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.523318446241319e-05 - rollout_corr/rollout_is_max:1.6244064569473267 - rollout_corr/rollout_is_min:0.396142840385437 - rollout_corr/rollout_is_std:0.03780318796634674 - rollout_corr/rollout_is_eff_sample_size:0.99857307726417 - rollout_corr/rollout_is_seq_mean:0.9999815225601196 - rollout_corr/rollout_is_seq_std:0.002393631497398019 - rollout_corr/rollout_is_seq_max:1.0126123428344727 - rollout_corr/rollout_is_seq_min:0.9941840171813965 - rollout_corr/rollout_is_seq_max_deviation:0.012612342834472656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.630201779305935) - rollout_corr/training_log_ppl:np.float64(0.47879417467629537) - rollout_corr/kl:np.float64(0.0010309481362873463) - rollout_corr/k3_kl:np.float64(0.0008061554526079817) - rollout_corr/rollout_ppl:np.float64(1.6284840498119593) - rollout_corr/rollout_log_ppl:np.float64(0.4777632247423753) - rollout_corr/log_ppl_diff:np.float64(0.0010309499339200556) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020579289994202554) - rollout_corr/log_ppl_diff_max:np.float64(0.007941722869873047) - rollout_corr/log_ppl_diff_min:np.float64(-0.0057179927825927734) - rollout_corr/ppl_ratio:np.float64(1.0010342469904572) - rollout_corr/chi2_token:np.float64(0.00116553227417171) - rollout_corr/chi2_seq:np.float64(0.674807739444077) - actor/pg_loss:np.float64(0.004637886886484921) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022305224865704076) - actor/ppo_kl:np.float64(0.0002447084712073888) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(5661.525033625308) - self_distillation/token_reweight_w_std:np.float64(108029.05970195204) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004851918201894) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08942344659590162) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.34092599898576736) - perf/mfu/actor:np.float64(0.04916186108901947) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.24573516845703) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027775494381785393 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027775494381785393 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:332.60546875 - response_length/max:929.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.60546875 - response_length_non_aborted/max:929.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5229528546333313e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2314282320439816) - timing_s/agent_loop/generate_sequences/max:np.float64(7.571410246193409) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8092300324497046) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.571410246193409) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:337 - timing_s/agent_loop/slowest/response_length:895 - timing_s/gen:14.387957438826561 - timing_s/reward:0.05958708934485912 - timing_s/old_log_prob:2.9241322185844183 - timing_s/adv:0.5914540458470583 - timing_s/update_actor:21.573941450566053 - timing_s/step:39.624077497050166 - timing_s/stop_profile:0.0001155603677034378 - timing_per_token_ms/update_actor:0.13227513013915507 - timing_per_token_ms/gen:0.16897785522480605 - timing_per_token_ms/adv:0.003626349921502022 - perf/total_num_tokens:163099 - perf/time_per_step:39.624077497050166 - perf/throughput:514.5198648856305 (TaskRunner pid=2255397) Training Progress: 55%|█████▌ | 55/100 [50:22<35:47, 47.71s/it] (TaskRunner pid=2255397) step:56 - global_seqlen/min:16165 - global_seqlen/max:23611 - global_seqlen/minmax_diff:7446 - global_seqlen/balanced_min:19603 - global_seqlen/balanced_max:19611 - global_seqlen/mean:19608.875 - actor/entropy:0.42714759707450867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2066904902458191 - training/rollout_probs_diff_mean:0.004840976092964411 - training/rollout_probs_diff_std:0.011059464886784554 - training/rollout_actor_probs_pearson_corr:0.9992184638977051 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0001850128173828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7041244506835938 - rollout_corr/rollout_is_min:0.5909921526908875 - rollout_corr/rollout_is_std:0.03720882534980774 - rollout_corr/rollout_is_eff_sample_size:0.9986178930118242 - rollout_corr/rollout_is_seq_mean:1.0002026557922363 - rollout_corr/rollout_is_seq_std:0.0023563196882605553 - rollout_corr/rollout_is_seq_max:1.0108153820037842 - rollout_corr/rollout_is_seq_min:0.9901830554008484 - rollout_corr/rollout_is_seq_max_deviation:0.01081538200378418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5864489898085594) - rollout_corr/training_log_ppl:np.float64(0.4489033415738959) - rollout_corr/kl:np.float64(0.0005826077157036202) - rollout_corr/k3_kl:np.float64(0.0008374885150033151) - rollout_corr/rollout_ppl:np.float64(1.5854855361394584) - rollout_corr/rollout_log_ppl:np.float64(0.4483207310840953) - rollout_corr/log_ppl_diff:np.float64(0.0005826104898005724) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019630271708592772) - rollout_corr/log_ppl_diff_max:np.float64(0.011552989482879639) - rollout_corr/log_ppl_diff_min:np.float64(-0.00930735468864441) - rollout_corr/ppl_ratio:np.float64(1.0005861730314791) - rollout_corr/chi2_token:np.float64(0.0022083548828959465) - rollout_corr/chi2_seq:np.float64(1.1513643299695104) - actor/pg_loss:np.float64(0.0053109838627278805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004679971348195977) - actor/ppo_kl:np.float64(4.2931282393965375e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(133.22875495348126) - self_distillation/token_reweight_w_std:np.float64(2217.535897645983) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020651405211538) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08932297839783132) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.3138376846909523) - perf/mfu/actor:np.float64(0.04696252700818672) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.17254257202148) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004433032590895891 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.004433032590895891 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:347.62109375 - response_length/max:909.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.62109375 - response_length_non_aborted/max:909.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.741447865962982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4314610064029694) - timing_s/agent_loop/generate_sequences/max:np.float64(7.775194456800818) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5458272197138285) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.775194456800818) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:230 - timing_s/agent_loop/slowest/response_length:909 - timing_s/gen:14.528320418670774 - timing_s/reward:0.0595672782510519 - timing_s/old_log_prob:2.9559899885207415 - timing_s/adv:0.5271639805287123 - timing_s/update_actor:21.64877647906542 - timing_s/step:39.80573454126716 - timing_s/stop_profile:2.855248749256134e-05 - timing_per_token_ms/update_actor:0.13800368760998158 - timing_per_token_ms/gen:0.16325606430617448 - timing_per_token_ms/adv:0.00336049352989853 - perf/total_num_tokens:156871 - perf/time_per_step:39.80573454126716 - perf/throughput:492.61432368924653 (TaskRunner pid=2255397) Training Progress: 56%|█████▌ | 56/100 [51:02<33:15, 45.35s/it] (TaskRunner pid=2255397) step:57 - global_seqlen/min:17187 - global_seqlen/max:22734 - global_seqlen/minmax_diff:5547 - global_seqlen/balanced_min:19426 - global_seqlen/balanced_max:19436 - global_seqlen/mean:19433.875 - actor/entropy:0.419066846370697 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647868514060974 - training/rollout_probs_diff_mean:0.005257493816316128 - training/rollout_probs_diff_std:0.012042389251291752 - training/rollout_actor_probs_pearson_corr:0.9990561008453369 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9998766779899597 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.874779187957756e-05 - rollout_corr/rollout_is_max:1.9517626762390137 - rollout_corr/rollout_is_min:0.412737101316452 - rollout_corr/rollout_is_std:0.039478376507759094 - rollout_corr/rollout_is_eff_sample_size:0.9984436453453855 - rollout_corr/rollout_is_seq_mean:0.9998682141304016 - rollout_corr/rollout_is_seq_std:0.002758133690804243 - rollout_corr/rollout_is_seq_max:1.010321855545044 - rollout_corr/rollout_is_seq_min:0.9932553768157959 - rollout_corr/rollout_is_seq_max_deviation:0.010321855545043945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5485623660497367) - rollout_corr/training_log_ppl:np.float64(0.4282529861666262) - rollout_corr/kl:np.float64(0.0008480568455153659) - rollout_corr/k3_kl:np.float64(0.0008880625406391118) - rollout_corr/rollout_ppl:np.float64(1.5472203940153122) - rollout_corr/rollout_log_ppl:np.float64(0.42740493212477304) - rollout_corr/log_ppl_diff:np.float64(0.0008480540418531746) - rollout_corr/log_ppl_abs_diff:np.float64(0.002164474077289924) - rollout_corr/log_ppl_diff_max:np.float64(0.007681757211685181) - rollout_corr/log_ppl_diff_min:np.float64(-0.008211076259613037) - rollout_corr/ppl_ratio:np.float64(1.0008519866969436) - rollout_corr/chi2_token:np.float64(0.0018828820902854204) - rollout_corr/chi2_seq:np.float64(1.0425616323482245) - actor/pg_loss:np.float64(0.002906394307501614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029778121393064794) - actor/ppo_kl:np.float64(-0.00014163734993388744) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(3090.494025749387) - self_distillation/token_reweight_w_std:np.float64(77747.68742029642) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994963144417852) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06993347342358902) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.22349200583994389) - perf/mfu/actor:np.float64(0.04662792309159809) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.20368194580078) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013542745728045702 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013542745728045702 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:320.52734375 - response_length/max:1046.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:320.52734375 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.3910775780677795e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5050442274659872) - timing_s/agent_loop/generate_sequences/max:np.float64(8.29741439782083) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6069312687322963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.29741439782083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:15.101491201668978 - timing_s/reward:0.05853153392672539 - timing_s/old_log_prob:2.997370980679989 - timing_s/adv:0.5432093199342489 - timing_s/update_actor:21.456135919317603 - timing_s/step:40.24460173584521 - timing_s/stop_profile:0.00010164454579353333 - timing_per_token_ms/update_actor:0.13800731917410708 - timing_per_token_ms/gen:0.18404108465869207 - timing_per_token_ms/adv:0.003493959130218812 - perf/total_num_tokens:155471 - perf/time_per_step:40.24460173584521 - perf/throughput:482.89395749419396 (TaskRunner pid=2255397) Training Progress: 57%|█████▋ | 57/100 [51:42<31:24, 43.83s/it] (TaskRunner pid=2255397) step:58 - global_seqlen/min:17591 - global_seqlen/max:22208 - global_seqlen/minmax_diff:4617 - global_seqlen/balanced_min:19936 - global_seqlen/balanced_max:19980 - global_seqlen/mean:19944.0 - actor/entropy:0.39805784821510315 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5443886518478394 - training/rollout_probs_diff_mean:0.0047958018258214 - training/rollout_probs_diff_std:0.011325978673994541 - training/rollout_actor_probs_pearson_corr:0.9991068243980408 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999314546585083 - rollout_corr/rollout_is_ratio_fraction_high:1.1276500117674004e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1276500117674004e-05 - rollout_corr/rollout_is_max:2.245145320892334 - rollout_corr/rollout_is_min:0.3792571425437927 - rollout_corr/rollout_is_std:0.037710048258304596 - rollout_corr/rollout_is_eff_sample_size:0.9985797339908784 - rollout_corr/rollout_is_seq_mean:0.9998851418495178 - rollout_corr/rollout_is_seq_std:0.002318696118891239 - rollout_corr/rollout_is_seq_max:1.007555365562439 - rollout_corr/rollout_is_seq_min:0.9894842505455017 - rollout_corr/rollout_is_seq_max_deviation:0.010515749454498291 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5350726060569286) - rollout_corr/training_log_ppl:np.float64(0.4149510355782695) - rollout_corr/kl:np.float64(0.0008157491595808253) - rollout_corr/k3_kl:np.float64(0.0008106242991630097) - rollout_corr/rollout_ppl:np.float64(1.533764130435884) - rollout_corr/rollout_log_ppl:np.float64(0.41413528152043) - rollout_corr/log_ppl_diff:np.float64(0.0008157540578395128) - rollout_corr/log_ppl_abs_diff:np.float64(0.00186522479634732) - rollout_corr/log_ppl_diff_max:np.float64(0.012087315320968628) - rollout_corr/log_ppl_diff_min:np.float64(-0.005733311176300049) - rollout_corr/ppl_ratio:np.float64(1.0008189915679395) - rollout_corr/chi2_token:np.float64(0.0016144579276442528) - rollout_corr/chi2_seq:np.float64(0.5739311659708619) - actor/pg_loss:np.float64(0.0032648365013301373) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039790487971913535) - actor/ppo_kl:np.float64(-6.927822899172043e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(20.40596836898476) - self_distillation/token_reweight_w_std:np.float64(398.28047909680754) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010924299713224) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060014044371200725) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.22483210638165474) - perf/mfu/actor:np.float64(0.048109296920814684) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.0834503173828) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00522806728258729 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00522806728258729 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.40625 - response_length/max:1083.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.40625 - response_length_non_aborted/max:1083.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012578070163726807 - timing_s/agent_loop/generate_sequences/min:np.float64(1.711790930479765) - timing_s/agent_loop/generate_sequences/max:np.float64(8.412942063063383) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.874052923943964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.412942063063383) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:1083 - timing_s/gen:14.847420612350106 - timing_s/reward:0.05969010666012764 - timing_s/old_log_prob:2.983126612380147 - timing_s/adv:0.5337398238480091 - timing_s/update_actor:21.578375332057476 - timing_s/step:40.09062473848462 - timing_s/stop_profile:0.00010734237730503082 - timing_per_token_ms/update_actor:0.1352435277029274 - timing_per_token_ms/gen:0.16742693518662727 - timing_per_token_ms/adv:0.003345240572653487 - perf/total_num_tokens:159552 - perf/time_per_step:40.09062473848462 - perf/throughput:497.4729161767076 (TaskRunner pid=2255397) Training Progress: 58%|█████▊ | 58/100 [52:22<29:54, 42.72s/it] (TaskRunner pid=2255397) step:59 - global_seqlen/min:16516 - global_seqlen/max:23158 - global_seqlen/minmax_diff:6642 - global_seqlen/balanced_min:19285 - global_seqlen/balanced_max:19288 - global_seqlen/mean:19286.5 - actor/entropy:0.39392659068107605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105931282043457 - training/rollout_probs_diff_mean:0.004885942675173283 - training/rollout_probs_diff_std:0.01138182170689106 - training/rollout_actor_probs_pearson_corr:0.9991214275360107 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:0.9998358488082886 - rollout_corr/rollout_is_ratio_fraction_high:1.1690437531797215e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1690437531797215e-05 - rollout_corr/rollout_is_max:3.892078161239624 - rollout_corr/rollout_is_min:0.4614630937576294 - rollout_corr/rollout_is_std:0.03779766708612442 - rollout_corr/rollout_is_eff_sample_size:0.9985728395255719 - rollout_corr/rollout_is_seq_mean:0.999951183795929 - rollout_corr/rollout_is_seq_std:0.002800993388518691 - rollout_corr/rollout_is_seq_max:1.0143204927444458 - rollout_corr/rollout_is_seq_min:0.9928278923034668 - rollout_corr/rollout_is_seq_max_deviation:0.0143204927444458 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5359033830463886) - rollout_corr/training_log_ppl:np.float64(0.4180586190195754) - rollout_corr/kl:np.float64(0.0011108192077600165) - rollout_corr/k3_kl:np.float64(0.0008730394356462057) - rollout_corr/rollout_ppl:np.float64(1.5341403023339808) - rollout_corr/rollout_log_ppl:np.float64(0.41694779935642146) - rollout_corr/log_ppl_diff:np.float64(0.0011108196631539613) - rollout_corr/log_ppl_abs_diff:np.float64(0.002291953394887969) - rollout_corr/log_ppl_diff_max:np.float64(0.008277744054794312) - rollout_corr/log_ppl_diff_min:np.float64(-0.007644861936569214) - rollout_corr/ppl_ratio:np.float64(1.0011151337530464) - rollout_corr/chi2_token:np.float64(0.0013092008884996176) - rollout_corr/chi2_seq:np.float64(0.4192327775526792) - actor/pg_loss:np.float64(0.0025457896990701556) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003396774759494292) - actor/ppo_kl:np.float64(0.0002558892238049282) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(9165.766619424336) - self_distillation/token_reweight_w_std:np.float64(181822.19756039116) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998559767846018) - self_distillation/token_reweight_w_clip_frac:np.float64(0.057435350317973644) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.21796750277280807) - perf/mfu/actor:np.float64(0.0472734036832577) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(198.940327167511) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026581131387501955 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026581131387501955 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.140625 - response_length/max:859.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.140625 - response_length_non_aborted/max:859.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001378711313009262 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6116437073796988) - timing_s/agent_loop/generate_sequences/max:np.float64(7.250096688047051) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7876410359021975) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.250096688047051) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:217 - timing_s/agent_loop/slowest/response_length:844 - timing_s/gen:14.891584478318691 - timing_s/reward:0.05919893458485603 - timing_s/old_log_prob:2.9019656274467707 - timing_s/adv:0.4909003674983978 - timing_s/update_actor:21.302226703613997 - timing_s/step:39.73218045942485 - timing_s/stop_profile:0.00011939182877540588 - timing_per_token_ms/update_actor:0.1380643630493739 - timing_per_token_ms/gen:0.17408913348513785 - timing_per_token_ms/adv:0.0031816320191480948 - perf/total_num_tokens:154292 - perf/time_per_step:39.73218045942485 - perf/throughput:485.41257431606823 (TaskRunner pid=2255397) Training Progress: 59%|█████▉ | 59/100 [53:02<28:35, 41.85s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains a nitrogen-containing ring with a substituent group: a chlorine-substituted benzene ring connected to a trifluoromethyl group (C(F)(F)F). This suggests that the correct reactant must contain a similar structure, specifically a nitrogen-containing ring with the same substituents. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrogen-containing ring with the exact substituents: a chlorine-substituted benzene ring and a trifluoromethyl group. This matches the structure of the product exactly. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option B: Contains a nitrogen but not the correct substituents. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option C: Contains a hydroxyl group and a different substituent, not matching the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option D: Contains a different structure with a carbon chain and a fluorinated group, not matching the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Therefore, the only correct reactant is Option A. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2255397) step:60 - global_seqlen/min:15969 - global_seqlen/max:20753 - global_seqlen/minmax_diff:4784 - global_seqlen/balanced_min:19004 - global_seqlen/balanced_max:19008 - global_seqlen/mean:19006.625 - actor/entropy:0.40371620655059814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8045270442962646 - training/rollout_probs_diff_mean:0.005058089271187782 - training/rollout_probs_diff_std:0.01212889701128006 - training/rollout_actor_probs_pearson_corr:0.9990420341491699 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9997034668922424 - rollout_corr/rollout_is_ratio_fraction_high:1.1908589840459172e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.763435936183669e-05 - rollout_corr/rollout_is_max:2.487139940261841 - rollout_corr/rollout_is_min:0.0640438050031662 - rollout_corr/rollout_is_std:0.03801152855157852 - rollout_corr/rollout_is_eff_sample_size:0.9985564358356739 - rollout_corr/rollout_is_seq_mean:0.9996296167373657 - rollout_corr/rollout_is_seq_std:0.0025690472684800625 - rollout_corr/rollout_is_seq_max:1.0076467990875244 - rollout_corr/rollout_is_seq_min:0.9901750683784485 - rollout_corr/rollout_is_seq_max_deviation:0.009824931621551514 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.534271144773811) - rollout_corr/training_log_ppl:np.float64(0.41600528117851354) - rollout_corr/kl:np.float64(0.001141798264093552) - rollout_corr/k3_kl:np.float64(0.000918725037195145) - rollout_corr/rollout_ppl:np.float64(1.5324818124063313) - rollout_corr/rollout_log_ppl:np.float64(0.4148634799348656) - rollout_corr/log_ppl_diff:np.float64(0.001141801243647933) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023173787049017847) - rollout_corr/log_ppl_diff_max:np.float64(0.009830474853515625) - rollout_corr/log_ppl_diff_min:np.float64(-0.006625443696975708) - rollout_corr/ppl_ratio:np.float64(1.0011462338734418) - rollout_corr/chi2_token:np.float64(0.0013863001950085163) - rollout_corr/chi2_seq:np.float64(0.6778646670281887) - actor/pg_loss:np.float64(0.005434742895886302) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011875206650984182) - actor/ppo_kl:np.float64(-4.705816970229648e-05) - actor/pg_clipfrac_lower:np.float64(2.3182492441264912e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(18888.49168556533) - self_distillation/token_reweight_w_std:np.float64(311593.95773308317) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030431225895882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10766072731348686) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.35879671573638916) - perf/mfu/actor:np.float64(0.045344773814958786) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.29048919677734) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16029546205916068) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7511857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13365319867771744) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6229333333333333) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13860952641542026) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6874666666666667) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16050258419670824) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8023714285714285) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10001117146576141) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5698666666666666) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10827654197635113) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.694952380952381) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12671450210804772) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8414666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07147207072829928) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5278238095238095) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07866088591113561) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7011380952380954) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09304988511758287) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8708) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0457160969007615) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49634285714285714) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05141404847998308) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7056047619047618) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06605128200501377) - val-aux/sciknoweval/score/mean@16:np.float64(0.6875) - val-aux/sciknoweval/score/std@16:np.float64(0.16029546205916068) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7511857142857143) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13365319867771744) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6229333333333333) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13860952641542026) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6874666666666667) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16050258419670824) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8023714285714285) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10001117146576141) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5698666666666666) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10827654197635113) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.694952380952381) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12671450210804772) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8414666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07147207072829928) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5278238095238095) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07866088591113561) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7011380952380954) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09304988511758287) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8708) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0457160969007615) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49634285714285714) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05141404847998308) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7056047619047618) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06605128200501377) - val-core/sciknoweval/acc/mean@16:np.float64(0.6875) - val-aux/sciknoweval/acc/std@16:np.float64(0.16029546205916068) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7511857142857143) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13365319867771744) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6229333333333333) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13860952641542026) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6874666666666667) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16050258419670824) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8023714285714285) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10001117146576141) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5698666666666666) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10827654197635113) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.694952380952381) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12671450210804772) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8414666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07147207072829928) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5278238095238095) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07866088591113561) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7011380952380954) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09304988511758287) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8708) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0457160969007615) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49634285714285714) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05141404847998308) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7056047619047618) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06605128200501377) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9982142857142857) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.006185546430642514) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998571428571429) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0017362747371820737) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9967380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.008056772590456738) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9983857142857143) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.005874140160911555) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00015050933932646775) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9938904761904761) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01020614644674161) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9993380952380951) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0038135892390515993) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9893857142857142) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.011570601968058545) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9998333333333334) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00189074176643801) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9839095238095237) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.01075169136568833) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00036325512888540755) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029056959319859743 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029056959319859743 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:328.01953125 - response_length/max:944.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:328.01953125 - response_length_non_aborted/max:944.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00029645487666130066 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6444534286856651) - timing_s/agent_loop/generate_sequences/max:np.float64(7.757375167682767) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.735986279869394) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.757375167682767) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:187 - timing_s/agent_loop/slowest/response_length:944 - timing_s/gen:14.574015535414219 - timing_s/reward:0.05933819338679314 - timing_s/old_log_prob:3.192529769614339 - timing_s/adv:0.5636057835072279 - timing_s/update_actor:21.57127202488482 - timing_s/step:40.06707218103111 - timing_s/testing:128.52822875231504 - timing_s/save_checkpoint:13.383056795224547 - timing_s/stop_profile:0.00014889799058437347 - timing_per_token_ms/update_actor:0.14186679660963494 - timing_per_token_ms/gen:0.17355597079316232 - timing_per_token_ms/adv:0.0037066403392713586 - perf/total_num_tokens:152053 - perf/time_per_step:40.06707218103111 - perf/throughput:474.3701989035844 (TaskRunner pid=2255397) Training Progress: 60%|██████ | 60/100 [56:05<56:00, 84.02s/it] (TaskRunner pid=2255397) step:61 - global_seqlen/min:15598 - global_seqlen/max:23498 - global_seqlen/minmax_diff:7900 - global_seqlen/balanced_min:19661 - global_seqlen/balanced_max:19666 - global_seqlen/mean:19664.0 - actor/entropy:0.375679075717926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102344214916229 - training/rollout_probs_diff_mean:0.00489032082259655 - training/rollout_probs_diff_std:0.011647011153399944 - training/rollout_actor_probs_pearson_corr:0.9990482330322266 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.00008225440979 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1082543096563313e-05 - rollout_corr/rollout_is_max:1.639464259147644 - rollout_corr/rollout_is_min:0.3959808051586151 - rollout_corr/rollout_is_std:0.03841473534703255 - rollout_corr/rollout_is_eff_sample_size:0.9985268392369275 - rollout_corr/rollout_is_seq_mean:1.000138759613037 - rollout_corr/rollout_is_seq_std:0.002528289332985878 - rollout_corr/rollout_is_seq_max:1.007035493850708 - rollout_corr/rollout_is_seq_min:0.993572473526001 - rollout_corr/rollout_is_seq_max_deviation:0.007035493850708008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4903184147551656) - rollout_corr/training_log_ppl:np.float64(0.3903155774460174) - rollout_corr/kl:np.float64(0.0006912411112880434) - rollout_corr/k3_kl:np.float64(0.0009254292584728319) - rollout_corr/rollout_ppl:np.float64(1.4893131870776415) - rollout_corr/rollout_log_ppl:np.float64(0.38962433370761573) - rollout_corr/log_ppl_diff:np.float64(0.0006912437384016812) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020783296204172075) - rollout_corr/log_ppl_diff_max:np.float64(0.008693814277648926) - rollout_corr/log_ppl_diff_min:np.float64(-0.006119638681411743) - rollout_corr/ppl_ratio:np.float64(1.0006948560476303) - rollout_corr/chi2_token:np.float64(0.002362349070608616) - rollout_corr/chi2_seq:np.float64(0.9650198624003679) - actor/pg_loss:np.float64(0.005624711047858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006064838689781027) - actor/ppo_kl:np.float64(1.730111737785478e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(5193.730784527026) - self_distillation/token_reweight_w_std:np.float64(104429.90018021135) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993646144866943) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08162092015845701) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.354709729552269) - perf/mfu/actor:np.float64(0.04738440538771457) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.02718353271484) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004777961410582066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004777961410582066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.46875 - response_length/max:838.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.46875 - response_length_non_aborted/max:838.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.24973326921463e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6630962695926428) - timing_s/agent_loop/generate_sequences/max:np.float64(7.389603292569518) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.984407690099033) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.389603292569518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:830 - timing_s/gen:13.70044944435358 - timing_s/reward:0.06281237117946148 - timing_s/old_log_prob:3.0475483629852533 - timing_s/adv:0.5286510195583105 - timing_s/update_actor:21.522440258413553 - timing_s/step:38.972098929807544 - timing_s/stop_profile:0.0001547839492559433 - timing_per_token_ms/update_actor:0.13681372214715695 - timing_per_token_ms/gen:0.15183581705330237 - timing_per_token_ms/adv:0.0033605257040677794 - perf/total_num_tokens:157312 - perf/time_per_step:38.972098929807544 - perf/throughput:504.5661008768538 (TaskRunner pid=2255397) Training Progress: 61%|██████ | 61/100 [56:44<45:50, 70.52s/it] (TaskRunner pid=2255397) step:62 - global_seqlen/min:17175 - global_seqlen/max:22516 - global_seqlen/minmax_diff:5341 - global_seqlen/balanced_min:19524 - global_seqlen/balanced_max:19535 - global_seqlen/mean:19531.875 - actor/entropy:0.4294959306716919 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20105724036693573 - training/rollout_probs_diff_mean:0.005539817735552788 - training/rollout_probs_diff_std:0.012213723734021187 - training/rollout_actor_probs_pearson_corr:0.9990189075469971 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000041723251343 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9759001731872559 - rollout_corr/rollout_is_min:0.5684848427772522 - rollout_corr/rollout_is_std:0.04096618667244911 - rollout_corr/rollout_is_eff_sample_size:0.9983247021636537 - rollout_corr/rollout_is_seq_mean:0.9999959468841553 - rollout_corr/rollout_is_seq_std:0.0028718190733343363 - rollout_corr/rollout_is_seq_max:1.008870244026184 - rollout_corr/rollout_is_seq_min:0.987141489982605 - rollout_corr/rollout_is_seq_max_deviation:0.01285851001739502 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.584152389317751) - rollout_corr/training_log_ppl:np.float64(0.4482707906863652) - rollout_corr/kl:np.float64(0.001287838817653153) - rollout_corr/k3_kl:np.float64(0.0013142074097913792) - rollout_corr/rollout_ppl:np.float64(1.5821097679436207) - rollout_corr/rollout_log_ppl:np.float64(0.4469829489826225) - rollout_corr/log_ppl_diff:np.float64(0.0012878417037427425) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026649965438991785) - rollout_corr/log_ppl_diff_max:np.float64(0.014283061027526855) - rollout_corr/log_ppl_diff_min:np.float64(-0.010016441345214844) - rollout_corr/ppl_ratio:np.float64(1.0012943353503942) - rollout_corr/chi2_token:np.float64(0.002664147410541773) - rollout_corr/chi2_seq:np.float64(0.6478703722823411) - actor/pg_loss:np.float64(0.005159451276995242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001237628416674852) - actor/ppo_kl:np.float64(0.0003923675134416271) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(9608.619164069882) - self_distillation/token_reweight_w_std:np.float64(147490.825859931) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000067490618676) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10762355217593722) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.41205330193042755) - perf/mfu/actor:np.float64(0.04771627973118822) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.9746551513672) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012106545269489288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012106545269489288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:311.12109375 - response_length/max:1015.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.12109375 - response_length_non_aborted/max:1015.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013557448983192444 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3771552108228207) - timing_s/agent_loop/generate_sequences/max:np.float64(7.987416280433536) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6242045445760596) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.987416280433536) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1015 - timing_s/gen:14.992265343666077 - timing_s/reward:0.06058591604232788 - timing_s/old_log_prob:2.887193411588669 - timing_s/adv:0.5111278612166643 - timing_s/update_actor:21.240903863683343 - timing_s/step:39.78005450963974 - timing_s/stop_profile:0.00012206099927425385 - timing_per_token_ms/update_actor:0.13593743472966205 - timing_per_token_ms/gen:0.18823389887461017 - timing_per_token_ms/adv:0.0032711136361502947 - perf/total_num_tokens:156255 - perf/time_per_step:39.78005450963974 - perf/throughput:490.99668768093113 (TaskRunner pid=2255397) Training Progress: 62%|██████▏ | 62/100 [57:23<38:49, 61.31s/it] (TaskRunner pid=2255397) step:63 - global_seqlen/min:15986 - global_seqlen/max:20551 - global_seqlen/minmax_diff:4565 - global_seqlen/balanced_min:18758 - global_seqlen/balanced_max:18761 - global_seqlen/mean:18759.625 - actor/entropy:0.35515502095222473 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24133962392807007 - training/rollout_probs_diff_mean:0.004971614107489586 - training/rollout_probs_diff_std:0.012004957534372807 - training/rollout_actor_probs_pearson_corr:0.9989368915557861 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001026391983032 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3019985999562778e-05 - rollout_corr/rollout_is_max:1.8590831756591797 - rollout_corr/rollout_is_min:0.4984112083911896 - rollout_corr/rollout_is_std:0.03878994286060333 - rollout_corr/rollout_is_eff_sample_size:0.998497719797901 - rollout_corr/rollout_is_seq_mean:1.000045895576477 - rollout_corr/rollout_is_seq_std:0.002819458954036236 - rollout_corr/rollout_is_seq_max:1.0079857110977173 - rollout_corr/rollout_is_seq_min:0.9885396957397461 - rollout_corr/rollout_is_seq_max_deviation:0.011460304260253906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4885337855666876) - rollout_corr/training_log_ppl:np.float64(0.38630677238688804) - rollout_corr/kl:np.float64(0.0008763675811707117) - rollout_corr/k3_kl:np.float64(0.0011061657141908654) - rollout_corr/rollout_ppl:np.float64(1.4872424700297415) - rollout_corr/rollout_log_ppl:np.float64(0.3854304051492363) - rollout_corr/log_ppl_diff:np.float64(0.0008763672376517206) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025414098927285522) - rollout_corr/log_ppl_diff_max:np.float64(0.016067981719970703) - rollout_corr/log_ppl_diff_min:np.float64(-0.010530859231948853) - rollout_corr/ppl_ratio:np.float64(1.0008826348930597) - rollout_corr/chi2_token:np.float64(0.002648768713697791) - rollout_corr/chi2_seq:np.float64(1.621614676201716) - actor/pg_loss:np.float64(0.004201351199299097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006837096566414402) - actor/ppo_kl:np.float64(9.11460165120559e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(328.2442679554224) - self_distillation/token_reweight_w_std:np.float64(5757.5124940559035) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0031547287944704) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0877059873600956) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.3450646325945854) - perf/mfu/actor:np.float64(0.04625351205637495) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.83617782592773) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001894407905638218 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001894407905638218 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.01953125 - response_length/max:723.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.01953125 - response_length_non_aborted/max:723.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013262778520584106 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4198575951159) - timing_s/agent_loop/generate_sequences/max:np.float64(6.652159977704287) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4228165260647074) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.652159977704287) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:723 - timing_s/gen:13.547540098428726 - timing_s/reward:0.06083318777382374 - timing_s/old_log_prob:2.9801845103502274 - timing_s/adv:0.5016210991889238 - timing_s/update_actor:21.087883727625012 - timing_s/step:38.26442988216877 - timing_s/stop_profile:0.00010959990322589874 - timing_per_token_ms/update_actor:0.1405137611201251 - timing_per_token_ms/gen:0.17638877805388617 - timing_per_token_ms/adv:0.003342424883152807 - perf/total_num_tokens:150077 - perf/time_per_step:38.26442988216877 - perf/throughput:490.26275989916127 (TaskRunner pid=2255397) Training Progress: 63%|██████▎ | 63/100 [58:02<33:33, 54.41s/it] (TaskRunner pid=2255397) step:64 - global_seqlen/min:15100 - global_seqlen/max:20599 - global_seqlen/minmax_diff:5499 - global_seqlen/balanced_min:18082 - global_seqlen/balanced_max:18090 - global_seqlen/mean:18088.125 - actor/entropy:0.3758540451526642 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24836724996566772 - training/rollout_probs_diff_mean:0.005264882929623127 - training/rollout_probs_diff_std:0.012344393879175186 - training/rollout_actor_probs_pearson_corr:0.9989309310913086 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000416040420532 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8669313905993477e-05 - rollout_corr/rollout_is_max:1.78631591796875 - rollout_corr/rollout_is_min:0.3966720402240753 - rollout_corr/rollout_is_std:0.04027950391173363 - rollout_corr/rollout_is_eff_sample_size:0.9983804272587137 - rollout_corr/rollout_is_seq_mean:1.0000438690185547 - rollout_corr/rollout_is_seq_std:0.0027673090808093548 - rollout_corr/rollout_is_seq_max:1.0073356628417969 - rollout_corr/rollout_is_seq_min:0.9913079738616943 - rollout_corr/rollout_is_seq_max_deviation:0.008692026138305664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.488935824483633) - rollout_corr/training_log_ppl:np.float64(0.3868299189489335) - rollout_corr/kl:np.float64(0.0007672079092806428) - rollout_corr/k3_kl:np.float64(0.0009736951585637144) - rollout_corr/rollout_ppl:np.float64(1.4878136315383017) - rollout_corr/rollout_log_ppl:np.float64(0.38606271165190265) - rollout_corr/log_ppl_diff:np.float64(0.0007672072970308363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023564034490846097) - rollout_corr/log_ppl_diff_max:np.float64(0.011275023221969604) - rollout_corr/log_ppl_diff_min:np.float64(-0.008923828601837158) - rollout_corr/ppl_ratio:np.float64(1.000771946972236) - rollout_corr/chi2_token:np.float64(0.002396456664428115) - rollout_corr/chi2_seq:np.float64(0.5857332549057901) - actor/pg_loss:np.float64(0.0037766702007502317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045624542644873145) - actor/ppo_kl:np.float64(-8.097473682511236e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(34646.944526499836) - self_distillation/token_reweight_w_std:np.float64(444681.0361109813) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998654320370406) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06305540914763696) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2998654544353485) - perf/mfu/actor:np.float64(0.04428300784041782) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.82562017440796) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009668726474046707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009668726474046707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.50390625 - response_length/max:914.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.50390625 - response_length_non_aborted/max:914.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010181218385696411 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3763788174837828) - timing_s/agent_loop/generate_sequences/max:np.float64(7.2169532254338264) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.232598840339051) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.2169532254338264) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:879 - timing_s/gen:14.560809908434749 - timing_s/reward:0.057523222640156746 - timing_s/old_log_prob:2.9381075855344534 - timing_s/adv:0.5264706443995237 - timing_s/update_actor:21.378882113844156 - timing_s/step:39.54835497029126 - timing_s/stop_profile:0.00011551380157470703 - timing_per_token_ms/update_actor:0.14774114311077127 - timing_per_token_ms/gen:0.2087242142233447 - timing_per_token_ms/adv:0.0036382339545939927 - perf/total_num_tokens:144705 - perf/time_per_step:39.54835497029126 - perf/throughput:457.3673168855647 (TaskRunner pid=2255397) Training Progress: 64%|██████▍ | 64/100 [58:41<29:58, 49.97s/it] (TaskRunner pid=2255397) step:65 - global_seqlen/min:15051 - global_seqlen/max:21236 - global_seqlen/minmax_diff:6185 - global_seqlen/balanced_min:18004 - global_seqlen/balanced_max:18008 - global_seqlen/mean:18006.625 - actor/entropy:0.3955051898956299 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1947164535522461 - training/rollout_probs_diff_mean:0.0055443341843783855 - training/rollout_probs_diff_std:0.012429744005203247 - training/rollout_actor_probs_pearson_corr:0.9989457726478577 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999876618385315 - rollout_corr/rollout_is_ratio_fraction_high:1.3995213521411642e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.031123399734497 - rollout_corr/rollout_is_min:0.504069447517395 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982950004968502 - rollout_corr/rollout_is_seq_mean:0.9999707341194153 - rollout_corr/rollout_is_seq_std:0.002948605688288808 - rollout_corr/rollout_is_seq_max:1.009615182876587 - rollout_corr/rollout_is_seq_min:0.9913778305053711 - rollout_corr/rollout_is_seq_max_deviation:0.009615182876586914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.525312386918813) - rollout_corr/training_log_ppl:np.float64(0.4125790528487414) - rollout_corr/kl:np.float64(0.00100548441882653) - rollout_corr/k3_kl:np.float64(0.001056205478960237) - rollout_corr/rollout_ppl:np.float64(1.5237079304642975) - rollout_corr/rollout_log_ppl:np.float64(0.4115735680970829) - rollout_corr/log_ppl_diff:np.float64(0.0010054847516585141) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024253172159660608) - rollout_corr/log_ppl_diff_max:np.float64(0.012961804866790771) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540702819824219) - rollout_corr/ppl_ratio:np.float64(1.0010107702109963) - rollout_corr/chi2_token:np.float64(0.0022052584681659937) - rollout_corr/chi2_seq:np.float64(0.9272577015217394) - actor/pg_loss:np.float64(0.003355647437274456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000539568421572767) - actor/ppo_kl:np.float64(7.455754089491506e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(178.37819157890044) - self_distillation/token_reweight_w_std:np.float64(3052.0747698339983) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989271517843008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07668937157723121) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.31669436022639275) - perf/mfu/actor:np.float64(0.04449943785630652) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.69052124023438) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013841266743838787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013841266743838787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:279.11328125 - response_length/max:754.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:279.11328125 - response_length_non_aborted/max:754.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.972788393497467e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2479094993323088) - timing_s/agent_loop/generate_sequences/max:np.float64(6.392431171610951) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2372494999872288) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.392431171610951) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:754 - timing_s/gen:13.095486616715789 - timing_s/reward:0.0594562403857708 - timing_s/old_log_prob:2.941354649141431 - timing_s/adv:0.5120643582195044 - timing_s/update_actor:20.915571719408035 - timing_s/step:37.61257514543831 - timing_s/stop_profile:0.00016906112432479858 - timing_per_token_ms/update_actor:0.14519358652307163 - timing_per_token_ms/gen:0.1832741328805759 - timing_per_token_ms/adv:0.003554694162700564 - perf/total_num_tokens:144053 - perf/time_per_step:37.61257514543831 - perf/throughput:478.7394888643741 (TaskRunner pid=2255397) Training Progress: 65%|██████▌ | 65/100 [59:19<26:59, 46.27s/it] (TaskRunner pid=2255397) step:66 - global_seqlen/min:16554 - global_seqlen/max:19900 - global_seqlen/minmax_diff:3346 - global_seqlen/balanced_min:18653 - global_seqlen/balanced_max:18704 - global_seqlen/mean:18662.625 - actor/entropy:0.3692823350429535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24191009998321533 - training/rollout_probs_diff_mean:0.005031523294746876 - training/rollout_probs_diff_std:0.011805592104792595 - training/rollout_actor_probs_pearson_corr:0.998999834060669 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999994039535522 - rollout_corr/rollout_is_ratio_fraction_high:1.3333866263565142e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2511892318725586 - rollout_corr/rollout_is_min:0.5517903566360474 - rollout_corr/rollout_is_std:0.03924669697880745 - rollout_corr/rollout_is_eff_sample_size:0.998462065654954 - rollout_corr/rollout_is_seq_mean:1.0000253915786743 - rollout_corr/rollout_is_seq_std:0.002698667347431183 - rollout_corr/rollout_is_seq_max:1.0073356628417969 - rollout_corr/rollout_is_seq_min:0.992901086807251 - rollout_corr/rollout_is_seq_max_deviation:0.007335662841796875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5085271298885345) - rollout_corr/training_log_ppl:np.float64(0.4001465483015636) - rollout_corr/kl:np.float64(0.0009092299301867168) - rollout_corr/k3_kl:np.float64(0.001055825624348472) - rollout_corr/rollout_ppl:np.float64(1.5071750050410628) - rollout_corr/rollout_log_ppl:np.float64(0.3992373197543202) - rollout_corr/log_ppl_diff:np.float64(0.0009092285472434014) - rollout_corr/log_ppl_abs_diff:np.float64(0.002248107542982325) - rollout_corr/log_ppl_diff_max:np.float64(0.008349627256393433) - rollout_corr/log_ppl_diff_min:np.float64(-0.008058488368988037) - rollout_corr/ppl_ratio:np.float64(1.0009135268628597) - rollout_corr/chi2_token:np.float64(0.0025282949209213257) - rollout_corr/chi2_seq:np.float64(0.6103038967121392) - actor/pg_loss:np.float64(0.002379831741563976) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033230165354325436) - actor/ppo_kl:np.float64(8.482311253033004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(14.794726703781635) - self_distillation/token_reweight_w_std:np.float64(315.56329661473865) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989614612422884) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04611051009851508) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.24986116960644722) - perf/mfu/actor:np.float64(0.044863735441974) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.69231033325195) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023150925990194082 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023150925990194082 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:292.95703125 - response_length/max:1040.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:292.95703125 - response_length_non_aborted/max:1040.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014759600162506104 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7828350365161896) - timing_s/agent_loop/generate_sequences/max:np.float64(8.071565691381693) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.388918655909947) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.071565691381693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:1040 - timing_s/gen:15.357409680262208 - timing_s/reward:0.0539607759565115 - timing_s/old_log_prob:2.9237958528101444 - timing_s/adv:0.5357848312705755 - timing_s/update_actor:21.778983931988478 - timing_s/step:40.74699264019728 - timing_s/stop_profile:0.00011424347758293152 - timing_per_token_ms/update_actor:0.14587299436700676 - timing_per_token_ms/gen:0.2047736533496301 - timing_per_token_ms/adv:0.0035886218529720196 - perf/total_num_tokens:149301 - perf/time_per_step:40.74699264019728 - perf/throughput:458.01232902742254 (TaskRunner pid=2255397) Training Progress: 66%|██████▌ | 66/100 [1:00:00<25:17, 44.63s/it] (TaskRunner pid=2255397) step:67 - global_seqlen/min:14936 - global_seqlen/max:21018 - global_seqlen/minmax_diff:6082 - global_seqlen/balanced_min:18548 - global_seqlen/balanced_max:18548 - global_seqlen/mean:18548.0 - actor/entropy:0.4010680615901947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18490827083587646 - training/rollout_probs_diff_mean:0.005276753567159176 - training/rollout_probs_diff_std:0.011970635503530502 - training/rollout_actor_probs_pearson_corr:0.9990053772926331 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0001569986343384 - rollout_corr/rollout_is_ratio_fraction_high:1.357367818854982e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.357367818854982e-05 - rollout_corr/rollout_is_max:2.239859104156494 - rollout_corr/rollout_is_min:0.4357939660549164 - rollout_corr/rollout_is_std:0.03935437649488449 - rollout_corr/rollout_is_eff_sample_size:0.9984541032441931 - rollout_corr/rollout_is_seq_mean:1.0001627206802368 - rollout_corr/rollout_is_seq_std:0.002580940490588546 - rollout_corr/rollout_is_seq_max:1.0100871324539185 - rollout_corr/rollout_is_seq_min:0.9937511682510376 - rollout_corr/rollout_is_seq_max_deviation:0.010087132453918457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5274818786419928) - rollout_corr/training_log_ppl:np.float64(0.41380137618398294) - rollout_corr/kl:np.float64(0.0010146739747582956) - rollout_corr/k3_kl:np.float64(0.0009835226737777703) - rollout_corr/rollout_ppl:np.float64(1.5259358463808894) - rollout_corr/rollout_log_ppl:np.float64(0.41278669840539806) - rollout_corr/log_ppl_diff:np.float64(0.0010146777785848826) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021784571872558445) - rollout_corr/log_ppl_diff_max:np.float64(0.014455139636993408) - rollout_corr/log_ppl_diff_min:np.float64(-0.0060721635818481445) - rollout_corr/ppl_ratio:np.float64(1.0010192152112722) - rollout_corr/chi2_token:np.float64(0.0019758078269660473) - rollout_corr/chi2_seq:np.float64(0.4387206360697746) - actor/pg_loss:np.float64(0.0029250025982037187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039720970880807727) - actor/ppo_kl:np.float64(0.0003566578742226767) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(547.347474707989) - self_distillation/token_reweight_w_std:np.float64(8326.106724163285) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982913499698043) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06155556399608031) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2857639491558075) - perf/mfu/actor:np.float64(0.04548089224569089) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.682861328125) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016373249236494303 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016373249236494303 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:287.78125 - response_length/max:673.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.78125 - response_length_non_aborted/max:673.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013326294720172882 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3205897863954306) - timing_s/agent_loop/generate_sequences/max:np.float64(5.870759850367904) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2840450326475548) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.870759850367904) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:673 - timing_s/gen:12.662429416552186 - timing_s/reward:0.05886896140873432 - timing_s/old_log_prob:2.852442506700754 - timing_s/adv:0.49736564978957176 - timing_s/update_actor:21.174225766211748 - timing_s/step:37.33046619966626 - timing_s/stop_profile:2.7060508728027344e-05 - timing_per_token_ms/update_actor:0.14269884735693705 - timing_per_token_ms/gen:0.1718757386327531 - timing_per_token_ms/adv:0.00335188194003108 - perf/total_num_tokens:148384 - perf/time_per_step:37.33046619966626 - perf/throughput:496.8595865048645 (TaskRunner pid=2255397) Training Progress: 67%|██████▋ | 67/100 [1:00:37<23:20, 42.45s/it] (TaskRunner pid=2255397) step:68 - global_seqlen/min:15394 - global_seqlen/max:21848 - global_seqlen/minmax_diff:6454 - global_seqlen/balanced_min:19111 - global_seqlen/balanced_max:19113 - global_seqlen/mean:19112.0 - actor/entropy:0.454579621553421 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294895052909851 - training/rollout_probs_diff_mean:0.005558677017688751 - training/rollout_probs_diff_std:0.011987541802227497 - training/rollout_actor_probs_pearson_corr:0.999093234539032 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.999990701675415 - rollout_corr/rollout_is_ratio_fraction_high:2.770083119685296e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.329737901687622 - rollout_corr/rollout_is_min:0.5801910758018494 - rollout_corr/rollout_is_std:0.04091522842645645 - rollout_corr/rollout_is_eff_sample_size:0.9983287417268669 - rollout_corr/rollout_is_seq_mean:0.9999210834503174 - rollout_corr/rollout_is_seq_std:0.0029238073620945215 - rollout_corr/rollout_is_seq_max:1.010511040687561 - rollout_corr/rollout_is_seq_min:0.9896543025970459 - rollout_corr/rollout_is_seq_max_deviation:0.010511040687561035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6329940822906792) - rollout_corr/training_log_ppl:np.float64(0.4769620407023467) - rollout_corr/kl:np.float64(0.0010583482470849503) - rollout_corr/k3_kl:np.float64(0.0009939358607766735) - rollout_corr/rollout_ppl:np.float64(1.631263299845159) - rollout_corr/rollout_log_ppl:np.float64(0.47590369096724316) - rollout_corr/log_ppl_diff:np.float64(0.0010583497351035476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024261822691187263) - rollout_corr/log_ppl_diff_max:np.float64(0.012225866317749023) - rollout_corr/log_ppl_diff_min:np.float64(-0.008922696113586426) - rollout_corr/ppl_ratio:np.float64(1.0010634786449373) - rollout_corr/chi2_token:np.float64(0.0018958586733788252) - rollout_corr/chi2_seq:np.float64(0.7360206483863294) - actor/pg_loss:np.float64(0.0039513889933004975) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004996858078811783) - actor/ppo_kl:np.float64(9.023317768352968e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(16211.263574679615) - self_distillation/token_reweight_w_std:np.float64(247760.0435439156) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013059759512544) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09038829631754197) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.35276055335998535) - perf/mfu/actor:np.float64(0.04664085001944063) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.4664192199707) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022922437638044357 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022922437638044357 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:282.03125 - response_length/max:777.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:282.03125 - response_length_non_aborted/max:777.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.814541459083557e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1940552610903978) - timing_s/agent_loop/generate_sequences/max:np.float64(6.472350174561143) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2505520836712094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.472350174561143) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:777 - timing_s/gen:13.590019196271896 - timing_s/reward:0.05382108315825462 - timing_s/old_log_prob:3.070949576795101 - timing_s/adv:0.5018815603107214 - timing_s/update_actor:21.207215555012226 - timing_s/step:38.45727911219001 - timing_s/stop_profile:2.8835609555244446e-05 - timing_per_token_ms/update_actor:0.13870353413439349 - timing_per_token_ms/gen:0.1882274126907465 - timing_per_token_ms/adv:0.003282502879805367 - perf/total_num_tokens:152896 - perf/time_per_step:38.45727911219001 - perf/throughput:496.9670356617083 (TaskRunner pid=2255397) Training Progress: 68%|██████▊ | 68/100 [1:01:16<22:00, 41.26s/it] (TaskRunner pid=2255397) step:69 - global_seqlen/min:17581 - global_seqlen/max:21958 - global_seqlen/minmax_diff:4377 - global_seqlen/balanced_min:20257 - global_seqlen/balanced_max:20260 - global_seqlen/mean:20258.25 - actor/entropy:0.46467649936676025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135446548461914 - training/rollout_probs_diff_mean:0.00523096090182662 - training/rollout_probs_diff_std:0.011185083538293839 - training/rollout_actor_probs_pearson_corr:0.999234676361084 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0000218152999878 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6324684619903564 - rollout_corr/rollout_is_min:0.5706769227981567 - rollout_corr/rollout_is_std:0.03825926035642624 - rollout_corr/rollout_is_eff_sample_size:0.998538606349212 - rollout_corr/rollout_is_seq_mean:1.0000791549682617 - rollout_corr/rollout_is_seq_std:0.002675000810995698 - rollout_corr/rollout_is_seq_max:1.0090395212173462 - rollout_corr/rollout_is_seq_min:0.989310622215271 - rollout_corr/rollout_is_seq_max_deviation:0.010689377784729004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.646835464052856) - rollout_corr/training_log_ppl:np.float64(0.4856712990440428) - rollout_corr/kl:np.float64(0.0008948077715515979) - rollout_corr/k3_kl:np.float64(0.0010960273730233894) - rollout_corr/rollout_ppl:np.float64(1.6454020515084267) - rollout_corr/rollout_log_ppl:np.float64(0.48477648955304176) - rollout_corr/log_ppl_diff:np.float64(0.0008948094910010695) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023694130359217525) - rollout_corr/log_ppl_diff_max:np.float64(0.012593626976013184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010994911193847656) - rollout_corr/ppl_ratio:np.float64(1.0008999034762383) - rollout_corr/chi2_token:np.float64(0.0027323926333338022) - rollout_corr/chi2_seq:np.float64(1.6403908263891935) - actor/pg_loss:np.float64(0.004149097250774503) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006782121126889251) - actor/ppo_kl:np.float64(0.0001744897412141455) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(112.03770920354873) - self_distillation/token_reweight_w_std:np.float64(1537.8677468675305) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990846202708781) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10416805295972154) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.3852573037147522) - perf/mfu/actor:np.float64(0.04884632323523585) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.50916624069214) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001025630161166191 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001025630161166191 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:323.2578125 - response_length/max:740.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:323.2578125 - response_length_non_aborted/max:740.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5704503059387207e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7049627471715212) - timing_s/agent_loop/generate_sequences/max:np.float64(6.463043302297592) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.702042857927154) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.463043302297592) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:225 - timing_s/agent_loop/slowest/response_length:740 - timing_s/gen:13.723623352125287 - timing_s/reward:0.06722629256546497 - timing_s/old_log_prob:3.223280254751444 - timing_s/adv:0.5192670281976461 - timing_s/update_actor:21.539511624723673 - timing_s/step:39.17174446769059 - timing_s/stop_profile:0.00010973773896694183 - timing_per_token_ms/update_actor:0.13290580149274786 - timing_per_token_ms/gen:0.16583637470243476 - timing_per_token_ms/adv:0.003204046673562907 - perf/total_num_tokens:162066 - perf/time_per_step:39.17174446769059 - perf/throughput:517.1648665458158 (TaskRunner pid=2255397) Training Progress: 69%|██████▉ | 69/100 [1:01:55<21:00, 40.65s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must analyze the structure and functional groups present in both the product and the options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar heterocycle). (TaskRunner pid=2255397) - A chlorinated aromatic ring (–Cl group). (TaskRunner pid=2255397) - A trifluoromethyl group (–C(F)(F)F). (TaskRunner pid=2255397) - A nitrogen atom bonded to a carbon chain that includes the chlorinated aromatic ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's examine the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: Contains a chlorinated aromatic ring, a trifluoromethyl group, and a nitrogen-containing ring with a nitrogen atom bonded to a carbon chain that includes the chlorinated aromatic ring. It closely matches the structure of the product and is a strong candidate. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: Contains a chlorinated aromatic ring but lacks the trifluoromethyl group and the correct nitrogen-containing ring structure. It is not a match. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: Contains a chlorinated aromatic ring and a trifluoromethyl group, but the nitrogen-containing ring is not correctly structured. It does not match the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: Contains a trifluoromethyl group and a carbon chain with a chlorinated aromatic ring, but the nitrogen-containing ring is not correctly structured. It also does not match the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Thus, **Option A** is the only one that contains all the necessary functional groups and structural features required to synthesize the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2255397) step:70 - global_seqlen/min:14508 - global_seqlen/max:21779 - global_seqlen/minmax_diff:7271 - global_seqlen/balanced_min:19295 - global_seqlen/balanced_max:19376 - global_seqlen/mean:19309.25 - actor/entropy:0.48265349864959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931375205516815 - training/rollout_probs_diff_mean:0.005450577940791845 - training/rollout_probs_diff_std:0.011540403589606285 - training/rollout_actor_probs_pearson_corr:0.9992024302482605 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0000245571136475 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6136518716812134 - rollout_corr/rollout_is_min:0.5989248752593994 - rollout_corr/rollout_is_std:0.03922086954116821 - rollout_corr/rollout_is_eff_sample_size:0.9984642048317262 - rollout_corr/rollout_is_seq_mean:1.0000163316726685 - rollout_corr/rollout_is_seq_std:0.002710202941671014 - rollout_corr/rollout_is_seq_max:1.0095508098602295 - rollout_corr/rollout_is_seq_min:0.9921322464942932 - rollout_corr/rollout_is_seq_max_deviation:0.009550809860229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6673680869862437) - rollout_corr/training_log_ppl:np.float64(0.4960062170866877) - rollout_corr/kl:np.float64(0.0012155248211556113) - rollout_corr/k3_kl:np.float64(0.0012827583032049006) - rollout_corr/rollout_ppl:np.float64(1.6654691863805056) - rollout_corr/rollout_log_ppl:np.float64(0.4947906881570816) - rollout_corr/log_ppl_diff:np.float64(0.00121552892960608) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026022420497611165) - rollout_corr/log_ppl_diff_max:np.float64(0.016035795211791992) - rollout_corr/log_ppl_diff_min:np.float64(-0.00719606876373291) - rollout_corr/ppl_ratio:np.float64(1.0012216775212437) - rollout_corr/chi2_token:np.float64(0.002924366621300578) - rollout_corr/chi2_seq:np.float64(1.3708638157695532) - actor/pg_loss:np.float64(0.0026991143822669983) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006329814200398687) - actor/ppo_kl:np.float64(0.0004067609955740181) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(1099.6124461798463) - self_distillation/token_reweight_w_std:np.float64(17668.276791150623) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005380015354604) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08962540709762834) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.30823376774787903) - perf/mfu/actor:np.float64(0.04660907978159935) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.58329391479492) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6967261904761904) - val-aux/sciknoweval/reward/std@16:np.float64(0.15836902316671747) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7580857142857141) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12446919093953822) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6347571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1415950973799327) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6969619047619046) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1573615579638144) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8046666666666668) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09049082267851498) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5808857142857142) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11888321084630435) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7069904761904763) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1207083032051214) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8401380952380952) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0621350178422806) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.533304761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09702903481875863) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7099809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08266567659967641) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8649238095238094) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.037984835978200744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4909904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07165352375405533) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7088047619047618) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.057017736779803374) - val-aux/sciknoweval/score/mean@16:np.float64(0.6967261904761904) - val-aux/sciknoweval/score/std@16:np.float64(0.15836902316671747) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7580857142857141) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12446919093953822) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6347571428571429) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1415950973799327) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6969619047619046) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1573615579638144) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8046666666666668) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09049082267851498) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5808857142857142) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11888321084630435) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7069904761904763) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1207083032051214) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8401380952380952) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0621350178422806) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.533304761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09702903481875863) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7099809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08266567659967641) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8649238095238094) - val-aux/sciknoweval/score/best@16/std:np.float64(0.037984835978200744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4909904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07165352375405533) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7088047619047618) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.057017736779803374) - val-core/sciknoweval/acc/mean@16:np.float64(0.6967261904761904) - val-aux/sciknoweval/acc/std@16:np.float64(0.15836902316671747) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7580857142857141) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12446919093953822) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6347571428571429) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1415950973799327) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6969619047619046) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1573615579638144) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8046666666666668) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09049082267851498) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5808857142857142) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11888321084630435) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7069904761904763) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1207083032051214) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8401380952380952) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0621350178422806) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.533304761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09702903481875863) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7099809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08266567659967641) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8649238095238094) - val-core/sciknoweval/acc/best@16/std:np.float64(0.037984835978200744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4909904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07165352375405533) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7088047619047618) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.057017736779803374) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.000300566358820181) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9993666666666667) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0016170225430592812) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9996809523809523) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011905809477718342) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.002042357581276089) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9998476190476191) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0008380952380952382) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980571428571428) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023403005795668832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999428571428571) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0005185012414787124) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9969666666666667) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002289832599412746) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011535261757671833 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011535261757671833 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.1953125 - response_length/max:1167.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.1953125 - response_length_non_aborted/max:1167.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000107618048787117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.561738733202219) - timing_s/agent_loop/generate_sequences/max:np.float64(8.901570096611977) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.487381806007761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.901570096611977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:1167 - timing_s/gen:15.990489525720477 - timing_s/reward:0.05796127952635288 - timing_s/old_log_prob:2.9395081773400307 - timing_s/adv:0.5521378461271524 - timing_s/update_actor:21.566690308973193 - timing_s/step:41.193913504481316 - timing_s/testing:129.7496616318822 - timing_s/save_checkpoint:13.531663544476032 - timing_s/stop_profile:0.00015689432621002197 - timing_per_token_ms/update_actor:0.13961372340311765 - timing_per_token_ms/gen:0.20466516735851117 - timing_per_token_ms/adv:0.0035743092438025328 - perf/total_num_tokens:154474 - perf/time_per_step:41.193913504481316 - perf/throughput:468.7403637408577 (TaskRunner pid=2255397) Training Progress: 70%|███████ | 70/100 [1:04:59<41:54, 83.82s/it] (TaskRunner pid=2255397) step:71 - global_seqlen/min:19243 - global_seqlen/max:22825 - global_seqlen/minmax_diff:3582 - global_seqlen/balanced_min:21239 - global_seqlen/balanced_max:21249 - global_seqlen/mean:21244.625 - actor/entropy:0.48243123292922974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1782420575618744 - training/rollout_probs_diff_mean:0.004945038352161646 - training/rollout_probs_diff_std:0.010490105487406254 - training/rollout_actor_probs_pearson_corr:0.9993298649787903 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0001386404037476 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.646081805229187 - rollout_corr/rollout_is_min:0.6092609167098999 - rollout_corr/rollout_is_std:0.036472681909799576 - rollout_corr/rollout_is_eff_sample_size:0.998671986340567 - rollout_corr/rollout_is_seq_mean:1.00009286403656 - rollout_corr/rollout_is_seq_std:0.0023478141520172358 - rollout_corr/rollout_is_seq_max:1.0064475536346436 - rollout_corr/rollout_is_seq_min:0.9918783903121948 - rollout_corr/rollout_is_seq_max_deviation:0.008121609687805176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6814591218717396) - rollout_corr/training_log_ppl:np.float64(0.5044128161389381) - rollout_corr/kl:np.float64(0.0009273746180333831) - rollout_corr/k3_kl:np.float64(0.0009558818163668548) - rollout_corr/rollout_ppl:np.float64(1.679951566271484) - rollout_corr/rollout_log_ppl:np.float64(0.5034854415280279) - rollout_corr/log_ppl_diff:np.float64(0.0009273746109101921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021467784827109426) - rollout_corr/log_ppl_diff_max:np.float64(0.009796887636184692) - rollout_corr/log_ppl_diff_min:np.float64(-0.007009267807006836) - rollout_corr/ppl_ratio:np.float64(1.0009312871843576) - rollout_corr/chi2_token:np.float64(0.002058735117316246) - rollout_corr/chi2_seq:np.float64(0.9684193118009716) - actor/pg_loss:np.float64(0.002505662152543664) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036506839978756034) - actor/ppo_kl:np.float64(0.0002983230811750559) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(1927.8832940307911) - self_distillation/token_reweight_w_std:np.float64(37339.10144847393) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009577083401382) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08062945469282568) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.5023428723216057) - perf/mfu/actor:np.float64(0.051698067264255104) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(201.09897232055664) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013479345943778753 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013479345943778753 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.51953125 - response_length/max:967.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.51953125 - response_length_non_aborted/max:967.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.134435534477234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8741174843162298) - timing_s/agent_loop/generate_sequences/max:np.float64(8.054633025079966) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1346120286834775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.054633025079966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:967 - timing_s/gen:14.367894859984517 - timing_s/reward:0.06300288252532482 - timing_s/old_log_prob:3.210338668897748 - timing_s/adv:0.5408081188797951 - timing_s/update_actor:21.329918820410967 - timing_s/step:39.616243589669466 - timing_s/stop_profile:8.219294250011444e-05 - timing_per_token_ms/update_actor:0.12550185529522742 - timing_per_token_ms/gen:0.15524635447151797 - timing_per_token_ms/adv:0.0031820290948875013 - perf/total_num_tokens:169957 - perf/time_per_step:39.616243589669466 - perf/throughput:536.260459725663 (TaskRunner pid=2255397) Training Progress: 71%|███████ | 71/100 [1:05:39<34:06, 70.57s/it] (TaskRunner pid=2255397) step:72 - global_seqlen/min:16380 - global_seqlen/max:21611 - global_seqlen/minmax_diff:5231 - global_seqlen/balanced_min:19858 - global_seqlen/balanced_max:19861 - global_seqlen/mean:19860.0 - actor/entropy:0.5429559946060181 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15169164538383484 - training/rollout_probs_diff_mean:0.005359958857297897 - training/rollout_probs_diff_std:0.010805261321365833 - training/rollout_actor_probs_pearson_corr:0.9993027448654175 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000475645065308 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.644222617149353 - rollout_corr/rollout_is_min:0.6008924245834351 - rollout_corr/rollout_is_std:0.03811253234744072 - rollout_corr/rollout_is_eff_sample_size:0.9985496605558143 - rollout_corr/rollout_is_seq_mean:0.9999592304229736 - rollout_corr/rollout_is_seq_std:0.0024577102158218622 - rollout_corr/rollout_is_seq_max:1.0064175128936768 - rollout_corr/rollout_is_seq_min:0.9917982220649719 - rollout_corr/rollout_is_seq_max_deviation:0.008201777935028076 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7471187515184283) - rollout_corr/training_log_ppl:np.float64(0.5459236325696111) - rollout_corr/kl:np.float64(0.0010623894131924772) - rollout_corr/k3_kl:np.float64(0.0008694290827975237) - rollout_corr/rollout_ppl:np.float64(1.745131317526102) - rollout_corr/rollout_log_ppl:np.float64(0.5448612435720861) - rollout_corr/log_ppl_diff:np.float64(0.0010623889975249767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021515846019610763) - rollout_corr/log_ppl_diff_max:np.float64(0.010035932064056396) - rollout_corr/log_ppl_diff_min:np.float64(-0.00523415207862854) - rollout_corr/ppl_ratio:np.float64(1.0010662653949112) - rollout_corr/chi2_token:np.float64(0.001382074085995555) - rollout_corr/chi2_seq:np.float64(0.6654008245095611) - actor/pg_loss:np.float64(0.005195157486014068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003912408333235362) - actor/ppo_kl:np.float64(0.0002666896097824889) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(7515.255449632183) - self_distillation/token_reweight_w_std:np.float64(120734.30576296372) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992036235053092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1153781593311578) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.3598603457212448) - perf/mfu/actor:np.float64(0.048463804802359345) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.67909240722656) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006963299238123 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006963299238123 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:333.78125 - response_length/max:799.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:333.78125 - response_length_non_aborted/max:799.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005877390503883362 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6762614008039236) - timing_s/agent_loop/generate_sequences/max:np.float64(6.705963581800461) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7834387093244004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.705963581800461) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:767 - timing_s/gen:14.802263040095568 - timing_s/reward:0.06146353296935558 - timing_s/old_log_prob:3.034574717283249 - timing_s/adv:0.524946503341198 - timing_s/update_actor:21.136499028652906 - timing_s/step:39.64624011516571 - timing_s/stop_profile:2.850964665412903e-05 - timing_per_token_ms/update_actor:0.13303435944519704 - timing_per_token_ms/gen:0.17323124052166894 - timing_per_token_ms/adv:0.003304043953557389 - perf/total_num_tokens:158880 - perf/time_per_step:39.64624011516571 - perf/throughput:500.9302254718232 (TaskRunner pid=2255397) Training Progress: 72%|███████▏ | 72/100 [1:06:19<28:36, 61.30s/it] (TaskRunner pid=2255397) step:73 - global_seqlen/min:19647 - global_seqlen/max:22925 - global_seqlen/minmax_diff:3278 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21202 - global_seqlen/mean:21199.0 - actor/entropy:0.5374614596366882 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.301607221364975 - training/rollout_probs_diff_mean:0.004932303912937641 - training/rollout_probs_diff_std:0.010089674964547157 - training/rollout_actor_probs_pearson_corr:0.9993661642074585 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999767541885376 - rollout_corr/rollout_is_ratio_fraction_high:1.021909702103585e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1998536586761475 - rollout_corr/rollout_is_min:0.6301840543746948 - rollout_corr/rollout_is_std:0.03611796349287033 - rollout_corr/rollout_is_eff_sample_size:0.9986969544688866 - rollout_corr/rollout_is_seq_mean:1.0000014305114746 - rollout_corr/rollout_is_seq_std:0.0019163991091772914 - rollout_corr/rollout_is_seq_max:1.0062246322631836 - rollout_corr/rollout_is_seq_min:0.9927696585655212 - rollout_corr/rollout_is_seq_max_deviation:0.00723034143447876 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7496894244104624) - rollout_corr/training_log_ppl:np.float64(0.5468992951209657) - rollout_corr/kl:np.float64(0.0007367991016540287) - rollout_corr/k3_kl:np.float64(0.0007361613289731395) - rollout_corr/rollout_ppl:np.float64(1.7483539679087698) - rollout_corr/rollout_log_ppl:np.float64(0.5461624904419295) - rollout_corr/log_ppl_diff:np.float64(0.0007368046790361404) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016733732772991061) - rollout_corr/log_ppl_diff_max:np.float64(0.009091198444366455) - rollout_corr/log_ppl_diff_min:np.float64(-0.005505651235580444) - rollout_corr/ppl_ratio:np.float64(1.0007393239066005) - rollout_corr/chi2_token:np.float64(0.0014872262254357338) - rollout_corr/chi2_seq:np.float64(0.43628761847503483) - actor/pg_loss:np.float64(0.0029139972757548094) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018539705752118607) - actor/ppo_kl:np.float64(6.380892117796577e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(1624.953690484399) - self_distillation/token_reweight_w_std:np.float64(32560.52424334915) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991668974980712) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06964316591620445) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.21447288990020752) - perf/mfu/actor:np.float64(0.050516435918862526) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.0999493598938) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007218515034765005 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007218515034765005 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.25 - response_length/max:996.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.25 - response_length_non_aborted/max:996.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8507784008979797e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6397324707359076) - timing_s/agent_loop/generate_sequences/max:np.float64(8.402389820665121) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.477701659416198) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.402389820665121) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:996 - timing_s/gen:15.686312183737755 - timing_s/reward:0.06267971731722355 - timing_s/old_log_prob:2.8734097313135862 - timing_s/adv:0.5590537320822477 - timing_s/update_actor:21.74830207787454 - timing_s/step:41.018087424337864 - timing_s/stop_profile:0.00012700818479061127 - timing_per_token_ms/update_actor:0.12823896220266606 - timing_per_token_ms/gen:0.16029995282596626 - timing_per_token_ms/adv:0.003296462876092314 - perf/total_num_tokens:169592 - perf/time_per_step:41.018087424337864 - perf/throughput:516.8207815418933 (TaskRunner pid=2255397) Training Progress: 73%|███████▎ | 73/100 [1:07:00<24:51, 55.23s/it] (TaskRunner pid=2255397) step:74 - global_seqlen/min:18287 - global_seqlen/max:25091 - global_seqlen/minmax_diff:6804 - global_seqlen/balanced_min:20904 - global_seqlen/balanced_max:20912 - global_seqlen/mean:20910.0 - actor/entropy:0.5127137303352356 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27330654859542847 - training/rollout_probs_diff_mean:0.004899736028164625 - training/rollout_probs_diff_std:0.010343815200030804 - training/rollout_actor_probs_pearson_corr:0.9993671774864197 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000099539756775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0278759873472154e-05 - rollout_corr/rollout_is_max:1.7854251861572266 - rollout_corr/rollout_is_min:0.48449382185935974 - rollout_corr/rollout_is_std:0.0365249402821064 - rollout_corr/rollout_is_eff_sample_size:0.9986679439991228 - rollout_corr/rollout_is_seq_mean:1.000190258026123 - rollout_corr/rollout_is_seq_std:0.0022744936868548393 - rollout_corr/rollout_is_seq_max:1.0073591470718384 - rollout_corr/rollout_is_seq_min:0.9917874336242676 - rollout_corr/rollout_is_seq_max_deviation:0.008212566375732422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7097132932394743) - rollout_corr/training_log_ppl:np.float64(0.5229829574818723) - rollout_corr/kl:np.float64(0.0006870645115757412) - rollout_corr/k3_kl:np.float64(0.0007633993515128168) - rollout_corr/rollout_ppl:np.float64(1.7085823002271354) - rollout_corr/rollout_log_ppl:np.float64(0.5222958918311633) - rollout_corr/log_ppl_diff:np.float64(0.0006870656507089734) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019122406374663115) - rollout_corr/log_ppl_diff_max:np.float64(0.008660197257995605) - rollout_corr/log_ppl_diff_min:np.float64(-0.005268260836601257) - rollout_corr/ppl_ratio:np.float64(1.000690019223839) - rollout_corr/chi2_token:np.float64(0.0016905386000871658) - rollout_corr/chi2_seq:np.float64(1.4013629674445838) - actor/pg_loss:np.float64(0.0029980618273839355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017913388501256122) - actor/ppo_kl:np.float64(0.00016305825718454514) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(15.748328928370029) - self_distillation/token_reweight_w_std:np.float64(289.3728471599752) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004254090599716) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06948782474501058) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2460213266313076) - perf/mfu/actor:np.float64(0.0504502030203242) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.804443359375) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005830626469105482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005830626469105482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:380.03125 - response_length/max:953.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:380.03125 - response_length_non_aborted/max:953.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.984895586967468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.763165319338441) - timing_s/agent_loop/generate_sequences/max:np.float64(8.103548426181078) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.310445136674389) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.103548426181078) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:953 - timing_s/gen:14.785902259871364 - timing_s/reward:0.07314063049852848 - timing_s/old_log_prob:2.9278498981148005 - timing_s/adv:0.5283288117498159 - timing_s/update_actor:21.43048808723688 - timing_s/step:39.835200319066644 - timing_s/stop_profile:0.00012897513806819916 - timing_per_token_ms/update_actor:0.1281114782833386 - timing_per_token_ms/gen:0.15198074027497085 - timing_per_token_ms/adv:0.0031583501419764222 - perf/total_num_tokens:167280 - perf/time_per_step:39.835200319066644 - perf/throughput:524.9126358727428 (TaskRunner pid=2255397) Training Progress: 74%|███████▍ | 74/100 [1:07:40<21:56, 50.63s/it] (TaskRunner pid=2255397) step:75 - global_seqlen/min:16481 - global_seqlen/max:26987 - global_seqlen/minmax_diff:10506 - global_seqlen/balanced_min:22439 - global_seqlen/balanced_max:22447 - global_seqlen/mean:22444.625 - actor/entropy:0.5642927289009094 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2400461733341217 - training/rollout_probs_diff_mean:0.0051166643388569355 - training/rollout_probs_diff_std:0.010140416212379932 - training/rollout_actor_probs_pearson_corr:0.9994109869003296 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999061822891235 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0068769370263908e-05 - rollout_corr/rollout_is_max:1.5713963508605957 - rollout_corr/rollout_is_min:0.4978005588054657 - rollout_corr/rollout_is_std:0.03701609745621681 - rollout_corr/rollout_is_eff_sample_size:0.9986314455154365 - rollout_corr/rollout_is_seq_mean:0.999901294708252 - rollout_corr/rollout_is_seq_std:0.0019736075773835182 - rollout_corr/rollout_is_seq_max:1.0060852766036987 - rollout_corr/rollout_is_seq_min:0.9931412935256958 - rollout_corr/rollout_is_seq_max_deviation:0.006858706474304199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.791621611919254) - rollout_corr/training_log_ppl:np.float64(0.570267406059429) - rollout_corr/kl:np.float64(0.0008821988329259511) - rollout_corr/k3_kl:np.float64(0.0008163573830870519) - rollout_corr/rollout_ppl:np.float64(1.7901061396114528) - rollout_corr/rollout_log_ppl:np.float64(0.56938520638505) - rollout_corr/log_ppl_diff:np.float64(0.0008821996743790805) - rollout_corr/log_ppl_abs_diff:np.float64(0.00185499427607283) - rollout_corr/log_ppl_diff_max:np.float64(0.00786745548248291) - rollout_corr/log_ppl_diff_min:np.float64(-0.005721062421798706) - rollout_corr/ppl_ratio:np.float64(1.000884982291609) - rollout_corr/chi2_token:np.float64(0.001510836649686098) - rollout_corr/chi2_seq:np.float64(0.9038489798549563) - actor/pg_loss:np.float64(0.002076247939839959) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023045666375764995) - actor/ppo_kl:np.float64(5.9166708297198056e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(6473.88584861136) - self_distillation/token_reweight_w_std:np.float64(110608.01085398125) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989379325415939) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0619521141925361) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.19757572747766972) - perf/mfu/actor:np.float64(0.053976922098505964) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.7879409790039) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480865199118853 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480865199118853 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:387.95703125 - response_length/max:918.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:387.95703125 - response_length_non_aborted/max:918.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013186223804950714 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1773367393761873) - timing_s/agent_loop/generate_sequences/max:np.float64(8.078486654907465) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.527875997897354) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.078486654907465) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:918 - timing_s/gen:14.925510486587882 - timing_s/reward:0.05783644877374172 - timing_s/old_log_prob:2.9777278639376163 - timing_s/adv:0.5886598229408264 - timing_s/update_actor:21.493691984564066 - timing_s/step:40.15656405314803 - timing_s/stop_profile:7.43325799703598e-05 - timing_per_token_ms/update_actor:0.11970400477042982 - timing_per_token_ms/gen:0.1502815277000703 - timing_per_token_ms/adv:0.003278400858450667 - perf/total_num_tokens:179557 - perf/time_per_step:40.15656405314803 - perf/throughput:558.9279244681911 (TaskRunner pid=2255397) Training Progress: 75%|███████▌ | 75/100 [1:08:20<19:47, 47.50s/it] (TaskRunner pid=2255397) step:76 - global_seqlen/min:20385 - global_seqlen/max:23735 - global_seqlen/minmax_diff:3350 - global_seqlen/balanced_min:22458 - global_seqlen/balanced_max:22461 - global_seqlen/mean:22459.875 - actor/entropy:0.5038673281669617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917871475219727 - training/rollout_probs_diff_mean:0.004544694907963276 - training/rollout_probs_diff_std:0.009781488217413425 - training/rollout_actor_probs_pearson_corr:0.9994357228279114 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.999882698059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.531444787979126 - rollout_corr/rollout_is_min:0.6298530101776123 - rollout_corr/rollout_is_std:0.03470920771360397 - rollout_corr/rollout_is_eff_sample_size:0.9987963637779437 - rollout_corr/rollout_is_seq_mean:0.9998737573623657 - rollout_corr/rollout_is_seq_std:0.001968828495591879 - rollout_corr/rollout_is_seq_max:1.0092756748199463 - rollout_corr/rollout_is_seq_min:0.9928333163261414 - rollout_corr/rollout_is_seq_max_deviation:0.009275674819946289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7229888113215566) - rollout_corr/training_log_ppl:np.float64(0.5273766612517647) - rollout_corr/kl:np.float64(0.0006758117173214506) - rollout_corr/k3_kl:np.float64(0.0006499772979395857) - rollout_corr/rollout_ppl:np.float64(1.72180867055431) - rollout_corr/rollout_log_ppl:np.float64(0.526700847200118) - rollout_corr/log_ppl_diff:np.float64(0.0006758140516467392) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016295489040203393) - rollout_corr/log_ppl_diff_max:np.float64(0.007842868566513062) - rollout_corr/log_ppl_diff_min:np.float64(-0.005845367908477783) - rollout_corr/ppl_ratio:np.float64(1.0006780109833926) - rollout_corr/chi2_token:np.float64(0.0012538495939224958) - rollout_corr/chi2_seq:np.float64(0.6266895108856261) - actor/pg_loss:np.float64(0.0024852192727848887) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022367794190358836) - actor/ppo_kl:np.float64(-8.031681899822729e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(11.586508150445297) - self_distillation/token_reweight_w_std:np.float64(205.42800806532614) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991163620725274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07528238961822353) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.21515963971614838) - perf/mfu/actor:np.float64(0.05377171741192728) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.86333084106445) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009749584714882076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009749584714882076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:416.18359375 - response_length/max:835.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:416.18359375 - response_length_non_aborted/max:835.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015104003250598907 - timing_s/agent_loop/generate_sequences/min:np.float64(1.69934743642807) - timing_s/agent_loop/generate_sequences/max:np.float64(7.779057312756777) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.804315655819664) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.779057312756777) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:835 - timing_s/gen:14.658727264031768 - timing_s/reward:0.06544599309563637 - timing_s/old_log_prob:3.1654940266162157 - timing_s/adv:0.5449143052101135 - timing_s/update_actor:21.7036225348711 - timing_s/step:40.22667368501425 - timing_s/stop_profile:0.00010753795504570007 - timing_per_token_ms/update_actor:0.1207910915291776 - timing_per_token_ms/gen:0.13758508080335421 - timing_per_token_ms/adv:0.003032710028495893 - perf/total_num_tokens:179679 - perf/time_per_step:40.22667368501425 - perf/throughput:558.3328906552629 (TaskRunner pid=2255397) Training Progress: 76%|███████▌ | 76/100 [1:09:00<18:08, 45.33s/it] (TaskRunner pid=2255397) step:77 - global_seqlen/min:19528 - global_seqlen/max:26165 - global_seqlen/minmax_diff:6637 - global_seqlen/balanced_min:22384 - global_seqlen/balanced_max:22391 - global_seqlen/mean:22389.375 - actor/entropy:0.5504021048545837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2723144292831421 - training/rollout_probs_diff_mean:0.005151774734258652 - training/rollout_probs_diff_std:0.01037499401718378 - training/rollout_actor_probs_pearson_corr:0.9993544816970825 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999526739120483 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6774260997772217 - rollout_corr/rollout_is_min:0.6362783908843994 - rollout_corr/rollout_is_std:0.037128645926713943 - rollout_corr/rollout_is_eff_sample_size:0.9986231237591031 - rollout_corr/rollout_is_seq_mean:0.9999860525131226 - rollout_corr/rollout_is_seq_std:0.002109500812366605 - rollout_corr/rollout_is_seq_max:1.0075759887695312 - rollout_corr/rollout_is_seq_min:0.993508517742157 - rollout_corr/rollout_is_seq_max_deviation:0.00757598876953125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7856992213055491) - rollout_corr/training_log_ppl:np.float64(0.5679072652128525) - rollout_corr/kl:np.float64(0.0009176925997458341) - rollout_corr/k3_kl:np.float64(0.0007757482836723284) - rollout_corr/rollout_ppl:np.float64(1.784037644509226) - rollout_corr/rollout_log_ppl:np.float64(0.5669895681203343) - rollout_corr/log_ppl_diff:np.float64(0.0009176970925182104) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019323080778121948) - rollout_corr/log_ppl_diff_max:np.float64(0.007754087448120117) - rollout_corr/log_ppl_diff_min:np.float64(-0.00577545166015625) - rollout_corr/ppl_ratio:np.float64(1.00092074717395) - rollout_corr/chi2_token:np.float64(0.00127337034791708) - rollout_corr/chi2_seq:np.float64(0.6404874103609473) - actor/pg_loss:np.float64(0.0034807646879926324) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002253228935842344) - actor/ppo_kl:np.float64(0.00018203446227005315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(130.1160973869264) - self_distillation/token_reweight_w_std:np.float64(2325.1732160223182) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002648527733982) - self_distillation/token_reweight_w_clip_frac:np.float64(0.091672543407185) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2861635349690914) - perf/mfu/actor:np.float64(0.053185240362257795) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.9845733642578) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007428093231283128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007428093231283128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.57421875 - response_length/max:872.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.57421875 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.42423939704895e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.215482749044895) - timing_s/agent_loop/generate_sequences/max:np.float64(7.765327328816056) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.492655532936624) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.765327328816056) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:368 - timing_s/agent_loop/slowest/response_length:857 - timing_s/gen:14.60610330849886 - timing_s/reward:0.06456474959850311 - timing_s/old_log_prob:2.9403065145015717 - timing_s/adv:0.5702805798500776 - timing_s/update_actor:21.948025058954954 - timing_s/step:40.21948102675378 - timing_s/stop_profile:0.00011724233627319336 - timing_per_token_ms/update_actor:0.12253594092596909 - timing_per_token_ms/gen:0.14913469923624767 - timing_per_token_ms/adv:0.003183879517907923 - perf/total_num_tokens:179115 - perf/time_per_step:40.21948102675378 - perf/throughput:556.6798583280254 (TaskRunner pid=2255397) Training Progress: 77%|███████▋ | 77/100 [1:09:40<16:47, 43.82s/it] (TaskRunner pid=2255397) step:78 - global_seqlen/min:20134 - global_seqlen/max:31492 - global_seqlen/minmax_diff:11358 - global_seqlen/balanced_min:22482 - global_seqlen/balanced_max:29618 - global_seqlen/mean:23385.375 - actor/entropy:0.48590248823165894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40464895963668823 - training/rollout_probs_diff_mean:0.004600349348038435 - training/rollout_probs_diff_std:0.009926710277795792 - training/rollout_actor_probs_pearson_corr:0.9993968605995178 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9998959302902222 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.656884347146843e-06 - rollout_corr/rollout_is_max:1.8092979192733765 - rollout_corr/rollout_is_min:0.48850229382514954 - rollout_corr/rollout_is_std:0.03525106608867645 - rollout_corr/rollout_is_eff_sample_size:0.9987587856840187 - rollout_corr/rollout_is_seq_mean:0.9998641014099121 - rollout_corr/rollout_is_seq_std:0.0020814789459109306 - rollout_corr/rollout_is_seq_max:1.0097109079360962 - rollout_corr/rollout_is_seq_min:0.9942805171012878 - rollout_corr/rollout_is_seq_max_deviation:0.009710907936096191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7207122039981186) - rollout_corr/training_log_ppl:np.float64(0.5288956113836321) - rollout_corr/kl:np.float64(0.0008265634318824766) - rollout_corr/k3_kl:np.float64(0.0007540007430009155) - rollout_corr/rollout_ppl:np.float64(1.7192450258880854) - rollout_corr/rollout_log_ppl:np.float64(0.5280690497056639) - rollout_corr/log_ppl_diff:np.float64(0.0008265616779681295) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017743823991622776) - rollout_corr/log_ppl_diff_max:np.float64(0.009331107139587402) - rollout_corr/log_ppl_diff_min:np.float64(-0.004447638988494873) - rollout_corr/ppl_ratio:np.float64(1.0008291078265756) - rollout_corr/chi2_token:np.float64(0.0013703966978937387) - rollout_corr/chi2_seq:np.float64(0.8413228269200772) - actor/pg_loss:np.float64(0.004617663682438433) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030698799764650175) - actor/ppo_kl:np.float64(-3.2745020313029727e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(38.89375674771145) - self_distillation/token_reweight_w_std:np.float64(700.807301894878) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001390076475218) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09478579316055402) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.28953084722161293) - perf/mfu/actor:np.float64(0.0530806510376341) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.00171661376953) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.025150412693619728 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.025150412693619728 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.23046875 - response_length/max:8192.0 - response_length/min:163.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:451.23046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.3435564041137695e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.152110392227769) - timing_s/agent_loop/generate_sequences/max:np.float64(59.49123938381672) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.095185146856238) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.49123938381672) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:398 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.28803966008127 - timing_s/reward:0.0661248117685318 - timing_s/old_log_prob:3.3414545375853777 - timing_s/adv:0.5907729119062424 - timing_s/update_actor:23.06786256469786 - timing_s/step:93.44836739264429 - timing_s/stop_profile:0.00011118501424789429 - timing_per_token_ms/update_actor:0.12330282582970052 - timing_per_token_ms/gen:0.5738478955986778 - timing_per_token_ms/adv:0.003157811837025504 - perf/total_num_tokens:187083 - perf/time_per_step:93.44836739264429 - perf/throughput:250.24915525534115 (TaskRunner pid=2255397) Training Progress: 78%|███████▊ | 78/100 [1:11:14<21:31, 58.72s/it] (TaskRunner pid=2255397) step:79 - global_seqlen/min:20379 - global_seqlen/max:23902 - global_seqlen/minmax_diff:3523 - global_seqlen/balanced_min:22483 - global_seqlen/balanced_max:22828 - global_seqlen/mean:22531.875 - actor/entropy:0.47975319623947144 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.999380886554718 - training/rollout_probs_diff_mean:0.004774122033268213 - training/rollout_probs_diff_std:0.010841231793165207 - training/rollout_actor_probs_pearson_corr:0.9993086457252502 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999167919158936 - rollout_corr/rollout_is_ratio_fraction_high:8.872878424881492e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7745756849762984e-05 - rollout_corr/rollout_is_max:2.256157159805298 - rollout_corr/rollout_is_min:1.8565188497632334e-07 - rollout_corr/rollout_is_std:0.035090066492557526 - rollout_corr/rollout_is_eff_sample_size:0.9987699636823175 - rollout_corr/rollout_is_seq_mean:0.9999032020568848 - rollout_corr/rollout_is_seq_std:0.002013471210375428 - rollout_corr/rollout_is_seq_max:1.0075592994689941 - rollout_corr/rollout_is_seq_min:0.9934999942779541 - rollout_corr/rollout_is_seq_max_deviation:0.007559299468994141 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6734817675314844) - rollout_corr/training_log_ppl:np.float64(0.5028578019700944) - rollout_corr/kl:np.float64(0.0007867071553491201) - rollout_corr/k3_kl:np.float64(0.0008091878738127889) - rollout_corr/rollout_ppl:np.float64(1.6721355388872325) - rollout_corr/rollout_log_ppl:np.float64(0.5020710933895316) - rollout_corr/log_ppl_diff:np.float64(0.0007867085805628449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017737957823555917) - rollout_corr/log_ppl_diff_max:np.float64(0.022169530391693115) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566405296325684) - rollout_corr/ppl_ratio:np.float64(1.000790162710473) - rollout_corr/chi2_token:np.float64(0.0015391185879707336) - rollout_corr/chi2_seq:np.float64(1.4496739441528916) - actor/pg_loss:np.float64(0.005306927720084786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039829218349041184) - actor/ppo_kl:np.float64(-4.736486153222863e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(76.99012057809159) - self_distillation/token_reweight_w_std:np.float64(1541.8618622039212) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995152873452753) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11978079983964562) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2994599714875221) - perf/mfu/actor:np.float64(0.052974969811297) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.06073379516602) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009753511287271976 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009753511287271976 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.24609375 - response_length/max:1635.0 - response_length/min:188.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.24609375 - response_length_non_aborted/max:1635.0 - response_length_non_aborted/min:188.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010463036596775055 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4277297761291265) - timing_s/agent_loop/generate_sequences/max:np.float64(12.42059437930584) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.095716175761481) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.42059437930584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:234 - timing_s/agent_loop/slowest/response_length:1635 - timing_s/gen:19.97506350465119 - timing_s/reward:0.06424454413354397 - timing_s/old_log_prob:2.966035883873701 - timing_s/adv:0.6315709538757801 - timing_s/update_actor:21.955910548567772 - timing_s/step:45.67770639620721 - timing_s/stop_profile:3.654882311820984e-05 - timing_per_token_ms/update_actor:0.12180472413285497 - timing_per_token_ms/gen:0.17723630697187465 - timing_per_token_ms/adv:0.003503763856069347 - perf/total_num_tokens:180255 - perf/time_per_step:45.67770639620721 - perf/throughput:493.2794743361042 (TaskRunner pid=2255397) Training Progress: 79%|███████▉ | 79/100 [1:12:00<19:11, 54.82s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must analyze the structure and functional groups of the target molecule and compare it to the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The target molecule contains: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely an azole or similar). (TaskRunner pid=2255397) - A chlorinated aromatic ring (–Cl). (TaskRunner pid=2255397) - A trifluoromethyl group (–C(F)(F)F). (TaskRunner pid=2255397) - A nitrogen-containing side chain that is connected to a carbon chain with a chlorinated aromatic group and a trifluoromethyl group. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, we analyze the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" (TaskRunner pid=2255397) - Contains a chlorinated aromatic ring (–Cl). (TaskRunner pid=2255397) - Contains a trifluoromethyl group (–C(F)(F)F). (TaskRunner pid=2255397) - Contains a nitrogen-containing ring with a side chain that connects to the chlorinated aromatic ring and the trifluoromethyl group. (TaskRunner pid=2255397) - Matches the structure of the target molecule closely. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" (TaskRunner pid=2255397) - Contains a chlorinated ring, but the side chain and functional groups do not match the target molecule. (TaskRunner pid=2255397) - The nitrogen-containing ring is different and does not include the trifluoromethyl group. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" (TaskRunner pid=2255397) - Contains a chlorinated aromatic ring and a trifluoromethyl group. (TaskRunner pid=2255397) - However, the functional group arrangement and the nitrogen-containing ring do not match the target molecule's structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" (TaskRunner pid=2255397) - Contains a trifluoromethyl group and a chlorinated ring. (TaskRunner pid=2255397) - However, the nitrogen-containing ring and the side chain are not structured correctly to match the target molecule. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion:** (TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and structural elements that match the target molecule "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2255397) step:80 - global_seqlen/min:21379 - global_seqlen/max:27736 - global_seqlen/minmax_diff:6357 - global_seqlen/balanced_min:24041 - global_seqlen/balanced_max:24046 - global_seqlen/mean:24044.0 - actor/entropy:0.5321111679077148 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22554826736450195 - training/rollout_probs_diff_mean:0.005127668380737305 - training/rollout_probs_diff_std:0.010481650941073895 - training/rollout_actor_probs_pearson_corr:0.999333381652832 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999201893806458 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.784020185470581 - rollout_corr/rollout_is_min:0.5773587822914124 - rollout_corr/rollout_is_std:0.03691612929105759 - rollout_corr/rollout_is_eff_sample_size:0.9986386974447631 - rollout_corr/rollout_is_seq_mean:0.9999967813491821 - rollout_corr/rollout_is_seq_std:0.002032036194577813 - rollout_corr/rollout_is_seq_max:1.0090333223342896 - rollout_corr/rollout_is_seq_min:0.99375981092453 - rollout_corr/rollout_is_seq_max_deviation:0.00903332233428955 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7447218438610435) - rollout_corr/training_log_ppl:np.float64(0.5451616903301328) - rollout_corr/kl:np.float64(0.0007389033336671069) - rollout_corr/k3_kl:np.float64(0.0007813431365661927) - rollout_corr/rollout_ppl:np.float64(1.743419658858329) - rollout_corr/rollout_log_ppl:np.float64(0.5444227863335982) - rollout_corr/log_ppl_diff:np.float64(0.0007389039965346456) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016596049536019564) - rollout_corr/log_ppl_diff_max:np.float64(0.005907297134399414) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042874813079833984) - rollout_corr/ppl_ratio:np.float64(1.0007411232218146) - rollout_corr/chi2_token:np.float64(0.001654008636251092) - rollout_corr/chi2_seq:np.float64(0.4709114315919578) - actor/pg_loss:np.float64(0.004708616062998772) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004071622186074819) - actor/ppo_kl:np.float64(2.530917598608795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(13418.10702541261) - self_distillation/token_reweight_w_std:np.float64(249965.62142065133) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001714508980513) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10478754635551013) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.277141023427248) - perf/mfu/actor:np.float64(0.05686537883473178) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.12682485580444) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7098214285714286) - val-aux/sciknoweval/reward/std@16:np.float64(0.14355997006676555) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7664952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10898051269342317) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6531333333333332) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13319323391175897) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7104857142857143) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14460069804004294) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8059285714285714) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07362869726753447) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6003619047619048) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11450096751541655) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7241619047619048) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10970231087023842) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8335476190476191) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05060468808975912) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5525523809523809) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0905650570350032) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7320380952380953) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07469910045616918) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8541285714285713) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.032361125070489466) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5139714285714285) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0607653237770218) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7368714285714285) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0520517104444629) - val-aux/sciknoweval/score/mean@16:np.float64(0.7098214285714286) - val-aux/sciknoweval/score/std@16:np.float64(0.14355997006676555) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7664952380952381) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10898051269342317) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6531333333333332) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13319323391175897) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7104857142857143) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14460069804004294) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8059285714285714) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07362869726753447) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6003619047619048) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11450096751541655) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7241619047619048) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10970231087023842) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8335476190476191) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05060468808975912) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5525523809523809) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0905650570350032) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7320380952380953) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07469910045616918) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8541285714285713) - val-aux/sciknoweval/score/best@16/std:np.float64(0.032361125070489466) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5139714285714285) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0607653237770218) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7368714285714285) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0520517104444629) - val-core/sciknoweval/acc/mean@16:np.float64(0.7098214285714286) - val-aux/sciknoweval/acc/std@16:np.float64(0.14355997006676555) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7664952380952381) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10898051269342317) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6531333333333332) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13319323391175897) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7104857142857143) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14460069804004294) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8059285714285714) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07362869726753447) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6003619047619048) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11450096751541655) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7241619047619048) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10970231087023842) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8335476190476191) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05060468808975912) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5525523809523809) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0905650570350032) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7320380952380953) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07469910045616918) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8541285714285713) - val-core/sciknoweval/acc/best@16/std:np.float64(0.032361125070489466) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5139714285714285) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0607653237770218) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7368714285714285) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0520517104444629) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9991071428571429) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0034580208448280513) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999428571428571) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0008968697662193748) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9982333333333334) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.004700687579230329) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9991285714285715) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0034158694143255103) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9967) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.006019070521218273) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9996714285714285) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.002136984020463583) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9942619047619048) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.007002323475158229) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999476190476191) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0008225042333482549) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9909714285714285) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006886708259921708) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0001505093393264677) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000642581726424396 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000642581726424396 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.6875 - response_length/max:1165.0 - response_length/min:201.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.6875 - response_length_non_aborted/max:1165.0 - response_length_non_aborted/min:201.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.976747393608093e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5183244552463293) - timing_s/agent_loop/generate_sequences/max:np.float64(9.816529558971524) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.047968282407965) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.816529558971524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:1165 - timing_s/gen:16.67618520371616 - timing_s/reward:0.06666623428463936 - timing_s/old_log_prob:3.0512595120817423 - timing_s/adv:0.615724191069603 - timing_s/update_actor:21.976544301956892 - timing_s/step:42.47468729875982 - timing_s/testing:132.552817100659 - timing_s/save_checkpoint:13.726791623979807 - timing_s/stop_profile:0.00016298145055770874 - timing_per_token_ms/update_actor:0.11425170677693443 - timing_per_token_ms/gen:0.14883072615054405 - timing_per_token_ms/adv:0.003201028276646996 - perf/total_num_tokens:192352 - perf/time_per_step:42.47468729875982 - perf/throughput:566.0783287438596 (TaskRunner pid=2255397) Training Progress: 80%|████████ | 80/100 [1:15:09<31:40, 95.02s/it] (TaskRunner pid=2255397) step:81 - global_seqlen/min:19063 - global_seqlen/max:27921 - global_seqlen/minmax_diff:8858 - global_seqlen/balanced_min:22607 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22615.0 - actor/entropy:0.4837610721588135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462881445884705 - training/rollout_probs_diff_mean:0.004895538557320833 - training/rollout_probs_diff_std:0.010500817559659481 - training/rollout_actor_probs_pearson_corr:0.9993140697479248 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999274611473083 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.790866897674277e-06 - rollout_corr/rollout_is_max:1.9132018089294434 - rollout_corr/rollout_is_min:0.4658448398113251 - rollout_corr/rollout_is_std:0.036682888865470886 - rollout_corr/rollout_is_eff_sample_size:0.998655936060176 - rollout_corr/rollout_is_seq_mean:0.9999848008155823 - rollout_corr/rollout_is_seq_std:0.002200948540121317 - rollout_corr/rollout_is_seq_max:1.0080451965332031 - rollout_corr/rollout_is_seq_min:0.9924934506416321 - rollout_corr/rollout_is_seq_max_deviation:0.008045196533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6707927286624908) - rollout_corr/training_log_ppl:np.float64(0.496672905923333) - rollout_corr/kl:np.float64(0.0007028676274636325) - rollout_corr/k3_kl:np.float64(0.0008774436023486487) - rollout_corr/rollout_ppl:np.float64(1.6695120050571859) - rollout_corr/rollout_log_ppl:np.float64(0.4959700393374078) - rollout_corr/log_ppl_diff:np.float64(0.0007028665859252214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018403253052383661) - rollout_corr/log_ppl_diff_max:np.float64(0.008645951747894287) - rollout_corr/log_ppl_diff_min:np.float64(-0.005466446280479431) - rollout_corr/ppl_ratio:np.float64(1.0007056852336973) - rollout_corr/chi2_token:np.float64(0.002068758476525545) - rollout_corr/chi2_seq:np.float64(0.5465720517095178) - actor/pg_loss:np.float64(0.0029664566973224282) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035411453472988796) - actor/ppo_kl:np.float64(-4.833459730235745e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(46.35069849877618) - self_distillation/token_reweight_w_std:np.float64(809.1466785562807) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000212090089917) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07078566148993559) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.2487758994102478) - perf/mfu/actor:np.float64(0.0539087676600922) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.76457595825195) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008355281315743923 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008355281315743923 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.96875 - response_length/max:1130.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.96875 - response_length_non_aborted/max:1130.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.358565926551819e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.753862602636218) - timing_s/agent_loop/generate_sequences/max:np.float64(8.9995809327811) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.538052094459999) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.9995809327811) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1130 - timing_s/gen:15.53846220485866 - timing_s/reward:0.06122434884309769 - timing_s/old_log_prob:3.0600726436823606 - timing_s/adv:0.5756961815059185 - timing_s/update_actor:21.892810126766562 - timing_s/step:41.233498902991414 - timing_s/stop_profile:0.00010543875396251678 - timing_per_token_ms/update_actor:0.12100823638495778 - timing_per_token_ms/gen:0.15213501806276591 - timing_per_token_ms/adv:0.0031820483169683754 - perf/total_num_tokens:180920 - perf/time_per_step:41.233498902991414 - perf/throughput:548.461823557722 (TaskRunner pid=2255397) Training Progress: 81%|████████ | 81/100 [1:15:50<24:59, 78.90s/it] (TaskRunner pid=2255397) step:82 - global_seqlen/min:20637 - global_seqlen/max:24388 - global_seqlen/minmax_diff:3751 - global_seqlen/balanced_min:22213 - global_seqlen/balanced_max:22222 - global_seqlen/mean:22220.125 - actor/entropy:0.44993990659713745 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135524034500122 - training/rollout_probs_diff_mean:0.0047300467267632484 - training/rollout_probs_diff_std:0.010585080832242966 - training/rollout_actor_probs_pearson_corr:0.9992892146110535 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999090433120728 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8240019926452078e-05 - rollout_corr/rollout_is_max:1.7895359992980957 - rollout_corr/rollout_is_min:0.000389088352676481 - rollout_corr/rollout_is_std:0.03580636531114578 - rollout_corr/rollout_is_eff_sample_size:0.9987193079476842 - rollout_corr/rollout_is_seq_mean:0.9998470544815063 - rollout_corr/rollout_is_seq_std:0.0019274608930572867 - rollout_corr/rollout_is_seq_max:1.0058091878890991 - rollout_corr/rollout_is_seq_min:0.9948349595069885 - rollout_corr/rollout_is_seq_max_deviation:0.005809187889099121 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6126991645433009) - rollout_corr/training_log_ppl:np.float64(0.4650081274448894) - rollout_corr/kl:np.float64(0.0009749815301454134) - rollout_corr/k3_kl:np.float64(0.0008126581537908351) - rollout_corr/rollout_ppl:np.float64(1.6110614440403879) - rollout_corr/rollout_log_ppl:np.float64(0.46403314505005255) - rollout_corr/log_ppl_diff:np.float64(0.000974982394836843) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017385364044457674) - rollout_corr/log_ppl_diff_max:np.float64(0.01138317584991455) - rollout_corr/log_ppl_diff_min:np.float64(-0.004374325275421143) - rollout_corr/ppl_ratio:np.float64(1.0009776803199202) - rollout_corr/chi2_token:np.float64(0.0012213955633342266) - rollout_corr/chi2_seq:np.float64(0.3682185474317521) - actor/pg_loss:np.float64(0.0047809198731556535) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044549755830303184) - actor/ppo_kl:np.float64(9.73689185599369e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(8763.06257886719) - self_distillation/token_reweight_w_std:np.float64(183308.58723631455) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998282515443861) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10694929157034494) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.27595510706305504) - perf/mfu/actor:np.float64(0.053128164780379857) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.69163131713867) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036035440862178802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036035440862178802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.31640625 - response_length/max:948.0 - response_length/min:190.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.31640625 - response_length_non_aborted/max:948.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010885484516620636 - timing_s/agent_loop/generate_sequences/min:np.float64(2.409843420609832) - timing_s/agent_loop/generate_sequences/max:np.float64(8.630525501444936) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.044243809614272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.630525501444936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:336 - timing_s/agent_loop/slowest/response_length:948 - timing_s/gen:15.308033773675561 - timing_s/reward:0.06256439350545406 - timing_s/old_log_prob:2.968469714745879 - timing_s/adv:0.5528020858764648 - timing_s/update_actor:21.71131898276508 - timing_s/step:40.68941228836775 - timing_s/stop_profile:3.694556653499603e-05 - timing_per_token_ms/update_actor:0.12213769602311575 - timing_per_token_ms/gen:0.13960942437847643 - timing_per_token_ms/adv:0.0031098052209228393 - perf/total_num_tokens:177761 - perf/time_per_step:40.68941228836775 - perf/throughput:546.0910775148323 (TaskRunner pid=2255397) Training Progress: 82%|████████▏ | 82/100 [1:16:31<20:13, 67.44s/it] (TaskRunner pid=2255397) step:83 - global_seqlen/min:17994 - global_seqlen/max:23375 - global_seqlen/minmax_diff:5381 - global_seqlen/balanced_min:21160 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21165.25 - actor/entropy:0.4580913484096527 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1788254976272583 - training/rollout_probs_diff_mean:0.005244065076112747 - training/rollout_probs_diff_std:0.011341297999024391 - training/rollout_actor_probs_pearson_corr:0.9992040395736694 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7578125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7578125 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6948878765106201 - rollout_corr/rollout_is_min:0.5044571757316589 - rollout_corr/rollout_is_std:0.03826860338449478 - rollout_corr/rollout_is_eff_sample_size:0.9985378931829969 - rollout_corr/rollout_is_seq_mean:0.9999917149543762 - rollout_corr/rollout_is_seq_std:0.0024205187801271677 - rollout_corr/rollout_is_seq_max:1.007947325706482 - rollout_corr/rollout_is_seq_min:0.9913325309753418 - rollout_corr/rollout_is_seq_max_deviation:0.008667469024658203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6229573977179825) - rollout_corr/training_log_ppl:np.float64(0.4732308021048084) - rollout_corr/kl:np.float64(0.0009288024647275961) - rollout_corr/k3_kl:np.float64(0.0008689218968811474) - rollout_corr/rollout_ppl:np.float64(1.6214703912846744) - rollout_corr/rollout_log_ppl:np.float64(0.47230199666228145) - rollout_corr/log_ppl_diff:np.float64(0.0009288054425269365) - rollout_corr/log_ppl_abs_diff:np.float64(0.002043650601990521) - rollout_corr/log_ppl_diff_max:np.float64(0.01018601655960083) - rollout_corr/log_ppl_diff_min:np.float64(-0.0065596699714660645) - rollout_corr/ppl_ratio:np.float64(1.0009324771817774) - rollout_corr/chi2_token:np.float64(0.0016192824114114046) - rollout_corr/chi2_seq:np.float64(0.7770539077464491) - actor/pg_loss:np.float64(0.005605395417660475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005950954305262712) - actor/ppo_kl:np.float64(0.00014756420982209306) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(43651.46375838155) - self_distillation/token_reweight_w_std:np.float64(663337.3462570815) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001920053968206) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12145704132854007) - self_distillation/empty_target_batch:np.float64(0.2421875) - actor/grad_norm:np.float64(0.37171755731105804) - perf/mfu/actor:np.float64(0.051307628909952094) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.40789794921875) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006963750929571688 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006963750929571688 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:362.5078125 - response_length/max:903.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:362.5078125 - response_length_non_aborted/max:903.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010122545063495636 - timing_s/agent_loop/generate_sequences/min:np.float64(1.972249450162053) - timing_s/agent_loop/generate_sequences/max:np.float64(8.24346206523478) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.390775169689732) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.24346206523478) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:903 - timing_s/gen:14.806242102757096 - timing_s/reward:0.06265864707529545 - timing_s/old_log_prob:3.0254632886499166 - timing_s/adv:0.5114385951310396 - timing_s/update_actor:21.41834749095142 - timing_s/step:39.9120793864131 - timing_s/stop_profile:0.00010847300291061401 - timing_per_token_ms/update_actor:0.12649477026583325 - timing_per_token_ms/gen:0.15954658415505157 - timing_per_token_ms/adv:0.0030205088241990977 - perf/total_num_tokens:169322 - perf/time_per_step:39.9120793864131 - perf/throughput:530.2968506122257 (TaskRunner pid=2255397) Training Progress: 83%|████████▎ | 83/100 [1:17:10<16:46, 59.20s/it] (TaskRunner pid=2255397) step:84 - global_seqlen/min:16326 - global_seqlen/max:23634 - global_seqlen/minmax_diff:7308 - global_seqlen/balanced_min:19956 - global_seqlen/balanced_max:19966 - global_seqlen/mean:19962.0 - actor/entropy:0.4355461001396179 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105639219284058 - training/rollout_probs_diff_mean:0.0052682701498270035 - training/rollout_probs_diff_std:0.011572654359042645 - training/rollout_actor_probs_pearson_corr:0.9991452097892761 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999979138374329 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7088184356689453 - rollout_corr/rollout_is_min:0.6109715700149536 - rollout_corr/rollout_is_std:0.038198452442884445 - rollout_corr/rollout_is_eff_sample_size:0.9985430042300492 - rollout_corr/rollout_is_seq_mean:0.999947190284729 - rollout_corr/rollout_is_seq_std:0.0022522625513374805 - rollout_corr/rollout_is_seq_max:1.0068590641021729 - rollout_corr/rollout_is_seq_min:0.9928377866744995 - rollout_corr/rollout_is_seq_max_deviation:0.007162213325500488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5662654708139598) - rollout_corr/training_log_ppl:np.float64(0.4385948544368148) - rollout_corr/kl:np.float64(0.0010095660052041877) - rollout_corr/k3_kl:np.float64(0.0008661396312845682) - rollout_corr/rollout_ppl:np.float64(1.5646833986975253) - rollout_corr/rollout_log_ppl:np.float64(0.4375852865341585) - rollout_corr/log_ppl_diff:np.float64(0.001009567902656272) - rollout_corr/log_ppl_abs_diff:np.float64(0.002128819382051006) - rollout_corr/log_ppl_diff_max:np.float64(0.010426610708236694) - rollout_corr/log_ppl_diff_min:np.float64(-0.0056244730949401855) - rollout_corr/ppl_ratio:np.float64(1.0010133925825357) - rollout_corr/chi2_token:np.float64(0.0014718861784785986) - rollout_corr/chi2_seq:np.float64(0.8280399544164538) - actor/pg_loss:np.float64(0.005005670711398125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003053363407161669) - actor/ppo_kl:np.float64(0.00018772196818872544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(1562.93544895784) - self_distillation/token_reweight_w_std:np.float64(29233.69878470525) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984272990841419) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09582373814191669) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.27201734483242035) - perf/mfu/actor:np.float64(0.04843784064306516) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.41874313354492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011461590183898807 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011461590183898807 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.78125 - response_length/max:898.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.78125 - response_length_non_aborted/max:898.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014925189316272736 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8480602335184813) - timing_s/agent_loop/generate_sequences/max:np.float64(7.672212071716785) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.319992882228689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.672212071716785) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:898 - timing_s/gen:14.744418485090137 - timing_s/reward:0.05119308829307556 - timing_s/old_log_prob:3.006163891404867 - timing_s/adv:0.5809233784675598 - timing_s/update_actor:21.276869893074036 - timing_s/step:39.69830561801791 - timing_s/stop_profile:2.6756897568702698e-05 - timing_per_token_ms/update_actor:0.13323358063491908 - timing_per_token_ms/gen:0.15575528696326096 - timing_per_token_ms/adv:0.0036376827125761436 - perf/total_num_tokens:159696 - perf/time_per_step:39.69830561801791 - perf/throughput:502.8426198356392 (TaskRunner pid=2255397) Training Progress: 84%|████████▍ | 84/100 [1:17:50<14:13, 53.36s/it] (TaskRunner pid=2255397) step:85 - global_seqlen/min:18150 - global_seqlen/max:26353 - global_seqlen/minmax_diff:8203 - global_seqlen/balanced_min:22089 - global_seqlen/balanced_max:22097 - global_seqlen/mean:22095.125 - actor/entropy:0.5015257596969604 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22547149658203125 - training/rollout_probs_diff_mean:0.005411901045590639 - training/rollout_probs_diff_std:0.011227205395698547 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000245571136475 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9998200514237396e-05 - rollout_corr/rollout_is_max:1.8690942525863647 - rollout_corr/rollout_is_min:0.3950064182281494 - rollout_corr/rollout_is_std:0.038822196424007416 - rollout_corr/rollout_is_eff_sample_size:0.9984952239250111 - rollout_corr/rollout_is_seq_mean:1.0000362396240234 - rollout_corr/rollout_is_seq_std:0.0021999897435307503 - rollout_corr/rollout_is_seq_max:1.0113379955291748 - rollout_corr/rollout_is_seq_min:0.9936926364898682 - rollout_corr/rollout_is_seq_max_deviation:0.011337995529174805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6718262596987188) - rollout_corr/training_log_ppl:np.float64(0.5036372784525156) - rollout_corr/kl:np.float64(0.0008671445817096668) - rollout_corr/k3_kl:np.float64(0.0008546028941509576) - rollout_corr/rollout_ppl:np.float64(1.670389695558697) - rollout_corr/rollout_log_ppl:np.float64(0.5027701300568879) - rollout_corr/log_ppl_diff:np.float64(0.000867148395627737) - rollout_corr/log_ppl_abs_diff:np.float64(0.00184359191916883) - rollout_corr/log_ppl_diff_max:np.float64(0.007470607757568359) - rollout_corr/log_ppl_diff_min:np.float64(-0.005036354064941406) - rollout_corr/ppl_ratio:np.float64(1.000870036194101) - rollout_corr/chi2_token:np.float64(0.001706940121948719) - rollout_corr/chi2_seq:np.float64(0.803199248155579) - actor/pg_loss:np.float64(0.0017579543637111783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.799221642315388e-05) - actor/ppo_kl:np.float64(0.00011437771347289072) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(531.5800662350375) - self_distillation/token_reweight_w_std:np.float64(6761.905646110186) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.998828696552664) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02915754570858553) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.16114498674869537) - perf/mfu/actor:np.float64(0.052797853671240076) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.15685653686523) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009824115550145507 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009824115550145507 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:390.66015625 - response_length/max:984.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:390.66015625 - response_length_non_aborted/max:984.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7002766728401184e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1650554779917) - timing_s/agent_loop/generate_sequences/max:np.float64(8.331863490864635) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.615080809744541) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.331863490864635) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:984 - timing_s/gen:15.139516476541758 - timing_s/reward:0.0630094688385725 - timing_s/old_log_prob:2.9453230183571577 - timing_s/adv:0.5465950388461351 - timing_s/update_actor:21.74274236895144 - timing_s/step:40.526393212378025 - timing_s/stop_profile:0.0001103375107049942 - timing_per_token_ms/update_actor:0.12300644581639299 - timing_per_token_ms/gen:0.15138154042677918 - timing_per_token_ms/adv:0.0030922830197053375 - perf/total_num_tokens:176761 - perf/time_per_step:40.526393212378025 - perf/throughput:545.2033415411727 (TaskRunner pid=2255397) Training Progress: 85%|████████▌ | 85/100 [1:18:31<12:22, 49.52s/it] (TaskRunner pid=2255397) step:86 - global_seqlen/min:19172 - global_seqlen/max:23707 - global_seqlen/minmax_diff:4535 - global_seqlen/balanced_min:21178 - global_seqlen/balanced_max:21185 - global_seqlen/mean:21182.625 - actor/entropy:0.4417911767959595 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772989869117737 - training/rollout_probs_diff_mean:0.004951337352395058 - training/rollout_probs_diff_std:0.011015241034328938 - training/rollout_actor_probs_pearson_corr:0.999230682849884 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0001946687698364 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0157130418519955e-05 - rollout_corr/rollout_is_max:1.913230538368225 - rollout_corr/rollout_is_min:0.48568272590637207 - rollout_corr/rollout_is_std:0.03722003847360611 - rollout_corr/rollout_is_eff_sample_size:0.9986170608525367 - rollout_corr/rollout_is_seq_mean:1.000307321548462 - rollout_corr/rollout_is_seq_std:0.0024071792140603065 - rollout_corr/rollout_is_seq_max:1.0088176727294922 - rollout_corr/rollout_is_seq_min:0.9944450259208679 - rollout_corr/rollout_is_seq_max_deviation:0.008817672729492188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.608684377744794) - rollout_corr/training_log_ppl:np.float64(0.4590825633786153) - rollout_corr/kl:np.float64(0.0005738776003347823) - rollout_corr/k3_kl:np.float64(0.0008272058389025005) - rollout_corr/rollout_ppl:np.float64(1.6077406979165971) - rollout_corr/rollout_log_ppl:np.float64(0.45850868528941646) - rollout_corr/log_ppl_diff:np.float64(0.0005738780891988426) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020339481707196683) - rollout_corr/log_ppl_diff_max:np.float64(0.008264422416687012) - rollout_corr/log_ppl_diff_min:np.float64(-0.009043440222740173) - rollout_corr/ppl_ratio:np.float64(1.00057721324265) - rollout_corr/chi2_token:np.float64(0.0023612575605511665) - rollout_corr/chi2_seq:np.float64(1.4658026506658643) - actor/pg_loss:np.float64(0.00446285936050117) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004066334154231299) - actor/ppo_kl:np.float64(0.00014794760728875644) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(132.06133578903973) - self_distillation/token_reweight_w_std:np.float64(2865.135968480725) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988646688871086) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1047386541031301) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.297544926404953) - perf/mfu/actor:np.float64(0.05125659080467514) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.21656799316406) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011623566970229149 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011623566970229149 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.58203125 - response_length/max:958.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:384.58203125 - response_length_non_aborted/max:958.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012109242379665375 - timing_s/agent_loop/generate_sequences/min:np.float64(2.020683040842414) - timing_s/agent_loop/generate_sequences/max:np.float64(8.49462386034429) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.405667071034259) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.49462386034429) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:958 - timing_s/gen:15.48012137785554 - timing_s/reward:0.06186640448868275 - timing_s/old_log_prob:2.9315725062042475 - timing_s/adv:0.5595468077808619 - timing_s/update_actor:21.52425908483565 - timing_s/step:40.646166034042835 - timing_s/stop_profile:0.00010807625949382782 - timing_per_token_ms/update_actor:0.12701600418288367 - timing_per_token_ms/gen:0.15723361784664297 - timing_per_token_ms/adv:0.0033019208418507023 - perf/total_num_tokens:169461 - perf/time_per_step:40.646166034042835 - perf/throughput:521.1469387360835 (TaskRunner pid=2255397) Training Progress: 86%|████████▌ | 86/100 [1:19:11<10:56, 46.87s/it] (TaskRunner pid=2255397) step:87 - global_seqlen/min:19881 - global_seqlen/max:22805 - global_seqlen/minmax_diff:2924 - global_seqlen/balanced_min:21579 - global_seqlen/balanced_max:21581 - global_seqlen/mean:21580.5 - actor/entropy:0.4654737114906311 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3751887083053589 - training/rollout_probs_diff_mean:0.0050046672113239765 - training/rollout_probs_diff_std:0.010888464748859406 - training/rollout_actor_probs_pearson_corr:0.9992407560348511 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0308853234164417e-05 - rollout_corr/rollout_is_max:1.854941487312317 - rollout_corr/rollout_is_min:0.3242693543434143 - rollout_corr/rollout_is_std:0.03734952583909035 - rollout_corr/rollout_is_eff_sample_size:0.998607075049263 - rollout_corr/rollout_is_seq_mean:1.0000066757202148 - rollout_corr/rollout_is_seq_std:0.00233814911916852 - rollout_corr/rollout_is_seq_max:1.0092549324035645 - rollout_corr/rollout_is_seq_min:0.9934799671173096 - rollout_corr/rollout_is_seq_max_deviation:0.009254932403564453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6343936491757631) - rollout_corr/training_log_ppl:np.float64(0.47929042315809056) - rollout_corr/kl:np.float64(0.0010372451878630784) - rollout_corr/k3_kl:np.float64(0.00085206899655077) - rollout_corr/rollout_ppl:np.float64(1.6326408791355789) - rollout_corr/rollout_log_ppl:np.float64(0.4782531792880036) - rollout_corr/log_ppl_diff:np.float64(0.001037243870086968) - rollout_corr/log_ppl_abs_diff:np.float64(0.002030018251389265) - rollout_corr/log_ppl_diff_max:np.float64(0.011031702160835266) - rollout_corr/log_ppl_diff_min:np.float64(-0.006544649600982666) - rollout_corr/ppl_ratio:np.float64(1.0010407408699393) - rollout_corr/chi2_token:np.float64(0.0013177578803151846) - rollout_corr/chi2_seq:np.float64(0.7552365991286933) - actor/pg_loss:np.float64(0.006197207723744214) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000917987231787265) - actor/ppo_kl:np.float64(0.00031247709554804715) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(8798.193575591547) - self_distillation/token_reweight_w_std:np.float64(137248.53065139498) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011895855423063) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1116473093861714) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3543427586555481) - perf/mfu/actor:np.float64(0.052530135237403436) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.04389572143555) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004306523595005274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004306523595005274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:378.921875 - response_length/max:813.0 - response_length/min:167.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.921875 - response_length_non_aborted/max:813.0 - response_length_non_aborted/min:167.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001322329044342041 - timing_s/agent_loop/generate_sequences/min:np.float64(2.250560261309147) - timing_s/agent_loop/generate_sequences/max:np.float64(7.587926117703319) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.51870351490652) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.587926117703319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:800 - timing_s/gen:14.453906053677201 - timing_s/reward:0.06278987228870392 - timing_s/old_log_prob:3.1354457326233387 - timing_s/adv:0.5562533307820559 - timing_s/update_actor:21.309262923896313 - timing_s/step:39.60505427233875 - timing_s/stop_profile:0.00011653825640678406 - timing_per_token_ms/update_actor:0.12342892266106156 - timing_per_token_ms/gen:0.14900319629785577 - timing_per_token_ms/adv:0.0032219673477332307 - perf/total_num_tokens:172644 - perf/time_per_step:39.60505427233875 - perf/throughput:544.8925748619012 (TaskRunner pid=2255397) Training Progress: 87%|████████▋ | 87/100 [1:19:51<09:41, 44.71s/it] (TaskRunner pid=2255397) step:88 - global_seqlen/min:18137 - global_seqlen/max:23978 - global_seqlen/minmax_diff:5841 - global_seqlen/balanced_min:21618 - global_seqlen/balanced_max:21626 - global_seqlen/mean:21623.75 - actor/entropy:0.4558359980583191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135497212409973 - training/rollout_probs_diff_mean:0.005039794836193323 - training/rollout_probs_diff_std:0.01103699766099453 - training/rollout_actor_probs_pearson_corr:0.9992441534996033 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.84765625 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84765625 - rollout_corr/rollout_is_mean:0.9999773502349854 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.879797284549568e-05 - rollout_corr/rollout_is_max:1.6429718732833862 - rollout_corr/rollout_is_min:0.34945860505104065 - rollout_corr/rollout_is_std:0.03734952583909035 - rollout_corr/rollout_is_eff_sample_size:0.9986068372944764 - rollout_corr/rollout_is_seq_mean:0.9999490976333618 - rollout_corr/rollout_is_seq_std:0.00212614587508142 - rollout_corr/rollout_is_seq_max:1.007338047027588 - rollout_corr/rollout_is_seq_min:0.9925395846366882 - rollout_corr/rollout_is_seq_max_deviation:0.007460415363311768 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6032785079441965) - rollout_corr/training_log_ppl:np.float64(0.46080948115559295) - rollout_corr/kl:np.float64(0.0011248015301781322) - rollout_corr/k3_kl:np.float64(0.0008873714502897201) - rollout_corr/rollout_ppl:np.float64(1.601451831869781) - rollout_corr/rollout_log_ppl:np.float64(0.4596846750937402) - rollout_corr/log_ppl_diff:np.float64(0.0011248060618527234) - rollout_corr/log_ppl_abs_diff:np.float64(0.002022003580350429) - rollout_corr/log_ppl_diff_max:np.float64(0.012501776218414307) - rollout_corr/log_ppl_diff_min:np.float64(-0.007802844047546387) - rollout_corr/ppl_ratio:np.float64(1.0011287301313132) - rollout_corr/chi2_token:np.float64(0.0012870903592556715) - rollout_corr/chi2_seq:np.float64(0.6421570053789765) - actor/pg_loss:np.float64(0.004205821780487895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004930502600473119) - actor/ppo_kl:np.float64(0.00033077471513121015) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(21107.031607169192) - self_distillation/token_reweight_w_std:np.float64(309542.6647549386) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036525861360133) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08997728512622416) - self_distillation/empty_target_batch:np.float64(0.15234375) - actor/grad_norm:np.float64(0.29456714540719986) - perf/mfu/actor:np.float64(0.05240681242673272) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.06198120117188) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007283487357199192 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007283487357199192 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.9296875 - response_length/max:943.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.9296875 - response_length_non_aborted/max:943.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010753795504570007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9089515656232834) - timing_s/agent_loop/generate_sequences/max:np.float64(8.329806737601757) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.679670039906341) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.329806737601757) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:943 - timing_s/gen:15.13785439543426 - timing_s/reward:0.06380515545606613 - timing_s/old_log_prob:3.0357294902205467 - timing_s/adv:0.5423688981682062 - timing_s/update_actor:21.611202096566558 - timing_s/step:40.480039244517684 - timing_s/stop_profile:0.00011552684009075165 - timing_per_token_ms/update_actor:0.12492746457348146 - timing_per_token_ms/gen:0.14531317214884962 - timing_per_token_ms/adv:0.0031352615652246152 - perf/total_num_tokens:172990 - perf/time_per_step:40.480039244517684 - perf/throughput:534.1830295514983 (TaskRunner pid=2255397) Training Progress: 88%|████████▊ | 88/100 [1:20:32<08:41, 43.45s/it] (TaskRunner pid=2255397) step:89 - global_seqlen/min:20192 - global_seqlen/max:26260 - global_seqlen/minmax_diff:6068 - global_seqlen/balanced_min:23178 - global_seqlen/balanced_max:23186 - global_seqlen/mean:23183.75 - actor/entropy:0.4386472702026367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7019415497779846 - training/rollout_probs_diff_mean:0.0047370451502501965 - training/rollout_probs_diff_std:0.010689225047826767 - training/rollout_actor_probs_pearson_corr:0.999261736869812 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999846816062927 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8036542314803228e-05 - rollout_corr/rollout_is_max:1.5554907321929932 - rollout_corr/rollout_is_min:0.03433068096637726 - rollout_corr/rollout_is_std:0.0357397198677063 - rollout_corr/rollout_is_eff_sample_size:0.9987241830413971 - rollout_corr/rollout_is_seq_mean:0.9999765157699585 - rollout_corr/rollout_is_seq_std:0.0020081165712326765 - rollout_corr/rollout_is_seq_max:1.0105236768722534 - rollout_corr/rollout_is_seq_min:0.9940011501312256 - rollout_corr/rollout_is_seq_max_deviation:0.010523676872253418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6203870358876884) - rollout_corr/training_log_ppl:np.float64(0.47031728448928334) - rollout_corr/kl:np.float64(0.0010279387461658018) - rollout_corr/k3_kl:np.float64(0.0008464122184363987) - rollout_corr/rollout_ppl:np.float64(1.6186228357255459) - rollout_corr/rollout_log_ppl:np.float64(0.4692893385363277) - rollout_corr/log_ppl_diff:np.float64(0.0010279459529556334) - rollout_corr/log_ppl_abs_diff:np.float64(0.00189788214629516) - rollout_corr/log_ppl_diff_max:np.float64(0.0077957212924957275) - rollout_corr/log_ppl_diff_min:np.float64(-0.004509687423706055) - rollout_corr/ppl_ratio:np.float64(1.0010310297366232) - rollout_corr/chi2_token:np.float64(0.0013277053367346525) - rollout_corr/chi2_seq:np.float64(0.5938672970514745) - actor/pg_loss:np.float64(0.0077381625305861235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008514890976130118) - actor/ppo_kl:np.float64(0.0002891868470378256) - actor/pg_clipfrac_lower:np.float64(8.172070920409169e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(47.27642107871361) - self_distillation/token_reweight_w_std:np.float64(801.2714838348329) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000362384133041) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13113477459410205) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3602307364344597) - perf/mfu/actor:np.float64(0.05579510012189607) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.08685684204102) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041562956757843494 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0041562956757843494 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.1484375 - response_length/max:1235.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.1484375 - response_length_non_aborted/max:1235.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.681098163127899e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.020791845396161) - timing_s/agent_loop/generate_sequences/max:np.float64(9.594862563535571) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.899216158031777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.594862563535571) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:223 - timing_s/agent_loop/slowest/response_length:1053 - timing_s/gen:16.61686352454126 - timing_s/reward:0.055103445425629616 - timing_s/old_log_prob:2.937300179153681 - timing_s/adv:0.5702405758202076 - timing_s/update_actor:21.570211308076978 - timing_s/step:41.78375179320574 - timing_s/stop_profile:2.8584152460098267e-05 - timing_per_token_ms/update_actor:0.11630027124643866 - timing_per_token_ms/gen:0.1498553787181543 - timing_per_token_ms/adv:0.003074570420122972 - perf/total_num_tokens:185470 - perf/time_per_step:41.78375179320574 - perf/throughput:554.8508452457781 (TaskRunner pid=2255397) Training Progress: 89%|████████▉ | 89/100 [1:21:13<07:52, 42.96s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure of the product and compare it with the options provided. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key features: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar structure). (TaskRunner pid=2255397) - A **chlorine-substituted benzene ring** (from the **Cl** in **"ccc(Cl)"**). (TaskRunner pid=2255397) - A **trifluoromethyl group** (from **"C(F)(F)F"**). (TaskRunner pid=2255397) - A **nitrogen-containing bridge** connecting the ring to the rest of the molecule. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let’s evaluate each option: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2255397) - Contains a **chlorine-substituted benzene ring**. (TaskRunner pid=2255397) - Contains a **trifluoromethyl group**. (TaskRunner pid=2255397) - Contains a **nitrogen-containing bridge** with the correct connectivity. (TaskRunner pid=2255397) - This matches the structure of the product and appears to be a direct precursor. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1` (TaskRunner pid=2255397) - Contains **chlorine atoms**, but the **trifluoromethyl group** is missing. (TaskRunner pid=2255397) - The **nitrogen-containing bridge** is not properly connected to the **trifluoromethyl** group. (TaskRunner pid=2255397) - This does not match the product structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` (TaskRunner pid=2255397) - Contains a **chlorine-substituted benzene ring** and a **trifluoromethyl group**. (TaskRunner pid=2255397) - However, the **nitrogen-containing bridge** is not properly positioned and is connected via an **oxygen**, which is not part of the product. (TaskRunner pid=2255397) - This is not a correct match. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F` (TaskRunner pid=2255397) - Contains a **trifluoromethyl group**. (TaskRunner pid=2255397) - However, it **lacks the chlorine-substituted benzene ring** and the **correct nitrogen-containing bridge**. (TaskRunner pid=2255397) - This also does not match the product structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion**: (TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and the correct connectivity to serve as the only correct reactant in the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2255397) step:90 - global_seqlen/min:18865 - global_seqlen/max:24932 - global_seqlen/minmax_diff:6067 - global_seqlen/balanced_min:22815 - global_seqlen/balanced_max:22831 - global_seqlen/mean:22826.625 - actor/entropy:0.40077415108680725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5775288939476013 - training/rollout_probs_diff_mean:0.0045609320513904095 - training/rollout_probs_diff_std:0.010716044344007969 - training/rollout_actor_probs_pearson_corr:0.9992610216140747 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9999971985816956 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.970459930424113e-06 - rollout_corr/rollout_is_max:1.8681867122650146 - rollout_corr/rollout_is_min:0.2344389706850052 - rollout_corr/rollout_is_std:0.03548699617385864 - rollout_corr/rollout_is_eff_sample_size:0.9987421380652595 - rollout_corr/rollout_is_seq_mean:1.0000590085983276 - rollout_corr/rollout_is_seq_std:0.0020531753543764353 - rollout_corr/rollout_is_seq_max:1.0109810829162598 - rollout_corr/rollout_is_seq_min:0.9949836730957031 - rollout_corr/rollout_is_seq_max_deviation:0.010981082916259766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5583982807584107) - rollout_corr/training_log_ppl:np.float64(0.43078241116018035) - rollout_corr/kl:np.float64(0.0008912494752149058) - rollout_corr/k3_kl:np.float64(0.000791456644748223) - rollout_corr/rollout_ppl:np.float64(1.5569715378805995) - rollout_corr/rollout_log_ppl:np.float64(0.42989116214448586) - rollout_corr/log_ppl_diff:np.float64(0.0008912490156944841) - rollout_corr/log_ppl_abs_diff:np.float64(0.001720420696074143) - rollout_corr/log_ppl_diff_max:np.float64(0.01338726282119751) - rollout_corr/log_ppl_diff_min:np.float64(-0.00902646780014038) - rollout_corr/ppl_ratio:np.float64(1.0008942331187427) - rollout_corr/chi2_token:np.float64(0.0014041787944734097) - rollout_corr/chi2_seq:np.float64(0.752974865725264) - actor/pg_loss:np.float64(0.004781530937179923) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005209917633237637) - actor/ppo_kl:np.float64(0.0002642208678480529) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(619.1751045542769) - self_distillation/token_reweight_w_std:np.float64(7680.31531262747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999175232835114) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1011948958330322) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.260063748806715) - perf/mfu/actor:np.float64(0.05435728539450956) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.9617691040039) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7092261904761905) - val-aux/sciknoweval/reward/std@16:np.float64(0.1437505073441559) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7665238095238094) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11153257282472556) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6511952380952379) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13026803458061142) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7090476190476193) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14272451357066823) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8092428571428572) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07643847293889101) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5996714285714285) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10642069734217646) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7207095238095238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11336797001466493) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8373142857142858) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.051272065112436704) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5569380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07817843871980924) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7271952380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08253191872957805) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8582761904761905) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03254491463909445) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.525004761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05193613577764248) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7298285714285715) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05986238009450025) - val-aux/sciknoweval/score/mean@16:np.float64(0.7092261904761905) - val-aux/sciknoweval/score/std@16:np.float64(0.1437505073441559) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7665238095238094) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11153257282472556) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6511952380952379) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13026803458061142) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7090476190476193) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14272451357066823) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8092428571428572) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07643847293889101) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5996714285714285) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10642069734217646) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7207095238095238) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11336797001466493) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8373142857142858) - val-aux/sciknoweval/score/best@8/std:np.float64(0.051272065112436704) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5569380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07817843871980924) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7271952380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08253191872957805) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8582761904761905) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03254491463909445) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.525004761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05193613577764248) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7298285714285715) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05986238009450025) - val-core/sciknoweval/acc/mean@16:np.float64(0.7092261904761905) - val-aux/sciknoweval/acc/std@16:np.float64(0.1437505073441559) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7665238095238094) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11153257282472556) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6511952380952379) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13026803458061142) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7090476190476193) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14272451357066823) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8092428571428572) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07643847293889101) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5996714285714285) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10642069734217646) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7207095238095238) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11336797001466493) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8373142857142858) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.051272065112436704) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5569380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07817843871980924) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7271952380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08253191872957805) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8582761904761905) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03254491463909445) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.525004761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05193613577764248) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7298285714285715) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05986238009450025) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9967261904761905) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.011218419246342346) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9996714285714285) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0035108104210424323) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9937666666666667) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.014896548825679245) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9966666666666667) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.011281400445573764) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0001505093393264677) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9886285714285715) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.018614743576868955) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9989571428571429) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.00637191435060849) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9802904761904762) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.02088499966775818) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997285714285713) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0030510338258627335) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9701857142857143) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.018986995237275702) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0003010186786529354) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0026496497448533773 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0026496497448533773 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:435.45703125 - response_length/max:1197.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:435.45703125 - response_length_non_aborted/max:1197.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.609938383102417e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6878780890256166) - timing_s/agent_loop/generate_sequences/max:np.float64(10.072803135961294) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.094866805477068) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.072803135961294) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:1197 - timing_s/gen:16.879685267806053 - timing_s/reward:0.06467907875776291 - timing_s/old_log_prob:3.10450641810894 - timing_s/adv:0.582694835960865 - timing_s/update_actor:21.818151073530316 - timing_s/step:42.53507168777287 - timing_s/testing:140.61721943691373 - timing_s/save_checkpoint:13.685923740267754 - timing_s/stop_profile:0.00018855556845664978 - timing_per_token_ms/update_actor:0.11947753486077287 - timing_per_token_ms/gen:0.15141854613782263 - timing_per_token_ms/adv:0.0031908726977863846 - perf/total_num_tokens:182613 - perf/time_per_step:42.53507168777287 - perf/throughput:536.6542030905226 (TaskRunner pid=2255397) Training Progress: 90%|█████████ | 90/100 [1:24:30<14:51, 89.15s/it] (TaskRunner pid=2255397) step:91 - global_seqlen/min:20110 - global_seqlen/max:28143 - global_seqlen/minmax_diff:8033 - global_seqlen/balanced_min:22954 - global_seqlen/balanced_max:22958 - global_seqlen/mean:22955.625 - actor/entropy:0.4345111846923828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.318805456161499 - training/rollout_probs_diff_mean:0.00483555207028985 - training/rollout_probs_diff_std:0.010917664505541325 - training/rollout_actor_probs_pearson_corr:0.9992064833641052 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000874996185303 - rollout_corr/rollout_is_ratio_fraction_high:9.058217074198183e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8116434148396365e-05 - rollout_corr/rollout_is_max:2.0618367195129395 - rollout_corr/rollout_is_min:0.27237245440483093 - rollout_corr/rollout_is_std:0.037339948117733 - rollout_corr/rollout_is_eff_sample_size:0.9986079071919081 - rollout_corr/rollout_is_seq_mean:1.0000126361846924 - rollout_corr/rollout_is_seq_std:0.0018744589760899544 - rollout_corr/rollout_is_seq_max:1.0062435865402222 - rollout_corr/rollout_is_seq_min:0.9926995038986206 - rollout_corr/rollout_is_seq_max_deviation:0.0073004961013793945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5809713881462812) - rollout_corr/training_log_ppl:np.float64(0.4482057260465808) - rollout_corr/kl:np.float64(0.0007782714664905654) - rollout_corr/k3_kl:np.float64(0.0007742295367734187) - rollout_corr/rollout_ppl:np.float64(1.5796857569366693) - rollout_corr/rollout_log_ppl:np.float64(0.4474274521926418) - rollout_corr/log_ppl_diff:np.float64(0.0007782738539390266) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016364274197258055) - rollout_corr/log_ppl_diff_max:np.float64(0.007638216018676758) - rollout_corr/log_ppl_diff_min:np.float64(-0.005470573902130127) - rollout_corr/ppl_ratio:np.float64(1.0007805426139385) - rollout_corr/chi2_token:np.float64(0.001570558873936534) - rollout_corr/chi2_seq:np.float64(0.9185826345346868) - actor/pg_loss:np.float64(0.004598198342137039) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000316593827392353) - actor/ppo_kl:np.float64(5.939174411295767e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(7563.319094766863) - self_distillation/token_reweight_w_std:np.float64(150023.32301567192) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999886890873313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08767912839539349) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.2757617086172104) - perf/mfu/actor:np.float64(0.054246744251757004) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(201.1460475921631) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008684565545991063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008684565545991063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:431.23828125 - response_length/max:1001.0 - response_length/min:187.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:431.23828125 - response_length_non_aborted/max:1001.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010878220200538635 - timing_s/agent_loop/generate_sequences/min:np.float64(2.29626976698637) - timing_s/agent_loop/generate_sequences/max:np.float64(8.867659989744425) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9120760589357815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.867659989744425) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:212 - timing_s/agent_loop/slowest/response_length:1001 - timing_s/gen:15.32518370822072 - timing_s/reward:0.061427073553204536 - timing_s/old_log_prob:3.1340453140437603 - timing_s/adv:0.5956883151084185 - timing_s/update_actor:22.10231014713645 - timing_s/step:41.29857017099857 - timing_s/stop_profile:0.00013570114970207214 - timing_per_token_ms/update_actor:0.12035345447540881 - timing_per_token_ms/gen:0.1388188420719831 - timing_per_token_ms/adv:0.003243694710492627 - perf/total_num_tokens:183645 - perf/time_per_step:41.29857017099857 - perf/throughput:555.8455148677353 (TaskRunner pid=2255397) Training Progress: 91%|█████████ | 91/100 [1:25:12<11:13, 74.81s/it] (TaskRunner pid=2255397) step:92 - global_seqlen/min:20808 - global_seqlen/max:26065 - global_seqlen/minmax_diff:5257 - global_seqlen/balanced_min:23344 - global_seqlen/balanced_max:23359 - global_seqlen/mean:23353.875 - actor/entropy:0.43259748816490173 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583375573158264 - training/rollout_probs_diff_mean:0.004716021008789539 - training/rollout_probs_diff_std:0.010604353621602058 - training/rollout_actor_probs_pearson_corr:0.9992642402648926 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0000042915344238 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.831503691908438e-06 - rollout_corr/rollout_is_max:1.6297050714492798 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.03597741574048996 - rollout_corr/rollout_is_eff_sample_size:0.9987072987735173 - rollout_corr/rollout_is_seq_mean:0.9999913573265076 - rollout_corr/rollout_is_seq_std:0.0019688971806317568 - rollout_corr/rollout_is_seq_max:1.0057662725448608 - rollout_corr/rollout_is_seq_min:0.9920455813407898 - rollout_corr/rollout_is_seq_max_deviation:0.007954418659210205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5944940163753927) - rollout_corr/training_log_ppl:np.float64(0.4538156447233632) - rollout_corr/kl:np.float64(0.0008374510896800302) - rollout_corr/k3_kl:np.float64(0.000751384576176406) - rollout_corr/rollout_ppl:np.float64(1.5930784032680094) - rollout_corr/rollout_log_ppl:np.float64(0.45297819224651903) - rollout_corr/log_ppl_diff:np.float64(0.0008374524768441916) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017089889151975513) - rollout_corr/log_ppl_diff_max:np.float64(0.008830234408378601) - rollout_corr/log_ppl_diff_min:np.float64(-0.005124390125274658) - rollout_corr/ppl_ratio:np.float64(1.0008400096558034) - rollout_corr/chi2_token:np.float64(0.0013158204965293407) - rollout_corr/chi2_seq:np.float64(0.8615647666156292) - actor/pg_loss:np.float64(0.00415497412905097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000329681670109494) - actor/ppo_kl:np.float64(0.000143634846812013) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(2081.352001107065) - self_distillation/token_reweight_w_std:np.float64(46560.20298913785) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010203949641436) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09130151267163455) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.25993651896715164) - perf/mfu/actor:np.float64(0.05560204307595709) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.7599754333496) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011590245179831982 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011590245179831982 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.30859375 - response_length/max:1140.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:442.30859375 - response_length_non_aborted/max:1140.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015668943524360657 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3020109608769417) - timing_s/agent_loop/generate_sequences/max:np.float64(9.68418238312006) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.095685060987307) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.68418238312006) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:1140 - timing_s/gen:16.275774881243706 - timing_s/reward:0.06520042568445206 - timing_s/old_log_prob:3.0083443708717823 - timing_s/adv:0.5682667158544064 - timing_s/update_actor:22.008645860478282 - timing_s/step:42.016618859022856 - timing_s/stop_profile:0.00013844482600688934 - timing_per_token_ms/update_actor:0.11779975411188873 - timing_per_token_ms/gen:0.1437395667374103 - timing_per_token_ms/adv:0.0030416082762197193 - perf/total_num_tokens:186831 - perf/time_per_step:42.016618859022856 - perf/throughput:555.8247101785744 (TaskRunner pid=2255397) Training Progress: 92%|█████████▏| 92/100 [1:25:54<08:39, 64.99s/it] (TaskRunner pid=2255397) step:93 - global_seqlen/min:20104 - global_seqlen/max:26536 - global_seqlen/minmax_diff:6432 - global_seqlen/balanced_min:22165 - global_seqlen/balanced_max:22172 - global_seqlen/mean:22170.375 - actor/entropy:0.4773474633693695 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463870882987976 - training/rollout_probs_diff_mean:0.005192984826862812 - training/rollout_probs_diff_std:0.01093792449682951 - training/rollout_actor_probs_pearson_corr:0.9992780685424805 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.9921875 - self_distillation/correct_sample_fraction:0.83203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.661463737487793 - rollout_corr/rollout_is_min:0.516041100025177 - rollout_corr/rollout_is_std:0.037531014531850815 - rollout_corr/rollout_is_eff_sample_size:0.9985935232071051 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0020964141003787518 - rollout_corr/rollout_is_seq_max:1.0067566633224487 - rollout_corr/rollout_is_seq_min:0.9933486580848694 - rollout_corr/rollout_is_seq_max_deviation:0.0067566633224487305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6435860991477966) - rollout_corr/training_log_ppl:np.float64(0.48362978905788623) - rollout_corr/kl:np.float64(0.000619826348660979) - rollout_corr/k3_kl:np.float64(0.0007837708870965798) - rollout_corr/rollout_ppl:np.float64(1.6425714306533337) - rollout_corr/rollout_log_ppl:np.float64(0.48300995968747884) - rollout_corr/log_ppl_diff:np.float64(0.0006198293704073876) - rollout_corr/log_ppl_abs_diff:np.float64(0.001762611762387678) - rollout_corr/log_ppl_diff_max:np.float64(0.009479671716690063) - rollout_corr/log_ppl_diff_min:np.float64(-0.005926311016082764) - rollout_corr/ppl_ratio:np.float64(1.0006224988028407) - rollout_corr/chi2_token:np.float64(0.0019295907113701105) - rollout_corr/chi2_seq:np.float64(1.5755573513451964) - actor/pg_loss:np.float64(0.004148329375311732) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002517454886401538) - actor/ppo_kl:np.float64(-5.375289406117645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_w_mean:np.float64(17544.475337664597) - self_distillation/token_reweight_w_std:np.float64(333058.533660894) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001345220953226) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08209344942588359) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.2505064271390438) - perf/mfu/actor:np.float64(0.05288764926923402) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.60185623168945) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00024851804482750595 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00024851804482750595 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:426.35546875 - response_length/max:917.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.35546875 - response_length_non_aborted/max:917.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014604441821575165 - timing_s/agent_loop/generate_sequences/min:np.float64(2.234377281740308) - timing_s/agent_loop/generate_sequences/max:np.float64(8.298982378095388) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.870763707032893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.298982378095388) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:917 - timing_s/gen:15.365983800962567 - timing_s/reward:0.06459944881498814 - timing_s/old_log_prob:2.9253345616161823 - timing_s/adv:0.5848103743046522 - timing_s/update_actor:22.019390346482396 - timing_s/step:41.04795786552131 - timing_s/stop_profile:0.00011224299669265747 - timing_per_token_ms/update_actor:0.1241487251934304 - timing_per_token_ms/gen:0.1407824658576284 - timing_per_token_ms/adv:0.0032972512547975183 - perf/total_num_tokens:177363 - perf/time_per_step:41.04795786552131 - perf/throughput:540.1090858803052 (TaskRunner pid=2255397) Training Progress: 93%|█████████▎| 93/100 [1:26:35<06:44, 57.82s/it] (TaskRunner pid=2255397) step:94 - global_seqlen/min:20450 - global_seqlen/max:26778 - global_seqlen/minmax_diff:6328 - global_seqlen/balanced_min:23452 - global_seqlen/balanced_max:23461 - global_seqlen/mean:23457.625 - actor/entropy:0.45124438405036926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47588256001472473 - training/rollout_probs_diff_mean:0.004685731139034033 - training/rollout_probs_diff_std:0.010418391786515713 - training/rollout_actor_probs_pearson_corr:0.999316930770874 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999880194664001 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4401948394370265e-05 - rollout_corr/rollout_is_max:1.8073612451553345 - rollout_corr/rollout_is_min:0.04583345353603363 - rollout_corr/rollout_is_std:0.03505437821149826 - rollout_corr/rollout_is_eff_sample_size:0.9987726987625546 - rollout_corr/rollout_is_seq_mean:0.9999796152114868 - rollout_corr/rollout_is_seq_std:0.0017362802755087614 - rollout_corr/rollout_is_seq_max:1.0060876607894897 - rollout_corr/rollout_is_seq_min:0.9951460361480713 - rollout_corr/rollout_is_seq_max_deviation:0.006087660789489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.600627651438117) - rollout_corr/training_log_ppl:np.float64(0.4611963724019006) - rollout_corr/kl:np.float64(0.0007944866175790821) - rollout_corr/k3_kl:np.float64(0.000699135587751698) - rollout_corr/rollout_ppl:np.float64(1.599346871022135) - rollout_corr/rollout_log_ppl:np.float64(0.46040188503684476) - rollout_corr/log_ppl_diff:np.float64(0.0007944873650558293) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015184025396592915) - rollout_corr/log_ppl_diff_max:np.float64(0.007160961627960205) - rollout_corr/log_ppl_diff_min:np.float64(-0.004008620977401733) - rollout_corr/ppl_ratio:np.float64(1.0007963897660375) - rollout_corr/chi2_token:np.float64(0.0012088804505765438) - rollout_corr/chi2_seq:np.float64(0.3547051209025085) - actor/pg_loss:np.float64(0.0030129868537187576) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016773132483649533) - actor/ppo_kl:np.float64(0.00010856679642934353) - actor/pg_clipfrac_lower:np.float64(7.659314178454224e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(4465.609730498167) - self_distillation/token_reweight_w_std:np.float64(81083.046648157) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998130949679762) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04856777528766543) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.18460461869835854) - perf/mfu/actor:np.float64(0.05546533287041886) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.40380096435547) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005895104259252548 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005895104259252548 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:480.23828125 - response_length/max:1044.0 - response_length/min:226.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.23828125 - response_length_non_aborted/max:1044.0 - response_length_non_aborted/min:226.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012932159006595612 - timing_s/agent_loop/generate_sequences/min:np.float64(2.888417324051261) - timing_s/agent_loop/generate_sequences/max:np.float64(9.281017616391182) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.676639520912431) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.281017616391182) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:997 - timing_s/gen:15.90216557495296 - timing_s/reward:0.06385539285838604 - timing_s/old_log_prob:2.9568253997713327 - timing_s/adv:0.5866717882454395 - timing_s/update_actor:21.823786733672023 - timing_s/step:41.424420485273004 - timing_s/stop_profile:0.00012981146574020386 - timing_per_token_ms/update_actor:0.11629367174677756 - timing_per_token_ms/gen:0.12934794393207277 - timing_per_token_ms/adv:0.0031262318129256454 - perf/total_num_tokens:187661 - perf/time_per_step:41.424420485273004 - perf/throughput:566.2752725373559 (TaskRunner pid=2255397) Training Progress: 94%|█████████▍| 94/100 [1:27:16<05:17, 52.92s/it] (TaskRunner pid=2255397) step:95 - global_seqlen/min:18176 - global_seqlen/max:26822 - global_seqlen/minmax_diff:8646 - global_seqlen/balanced_min:23320 - global_seqlen/balanced_max:23330 - global_seqlen/mean:23326.75 - actor/entropy:0.42938148975372314 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102376401424408 - training/rollout_probs_diff_mean:0.0046544442884624004 - training/rollout_probs_diff_std:0.010648963041603565 - training/rollout_actor_probs_pearson_corr:0.9992654323577881 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000568628311157 - rollout_corr/rollout_is_ratio_fraction_high:1.6773175957496278e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.386587978748139e-06 - rollout_corr/rollout_is_max:2.396075487136841 - rollout_corr/rollout_is_min:0.3959798514842987 - rollout_corr/rollout_is_std:0.035107050091028214 - rollout_corr/rollout_is_eff_sample_size:0.9987691312695665 - rollout_corr/rollout_is_seq_mean:1.0000056028366089 - rollout_corr/rollout_is_seq_std:0.0018295958871021867 - rollout_corr/rollout_is_seq_max:1.0062776803970337 - rollout_corr/rollout_is_seq_min:0.9954047203063965 - rollout_corr/rollout_is_seq_max_deviation:0.006277680397033691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5648417291231453) - rollout_corr/training_log_ppl:np.float64(0.4370654333033599) - rollout_corr/kl:np.float64(0.0006150017093133897) - rollout_corr/k3_kl:np.float64(0.0007087535628897967) - rollout_corr/rollout_ppl:np.float64(1.5638857381418347) - rollout_corr/rollout_log_ppl:np.float64(0.43645042926073074) - rollout_corr/log_ppl_diff:np.float64(0.0006150040426291525) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015464308089576662) - rollout_corr/log_ppl_diff_max:np.float64(0.005993485450744629) - rollout_corr/log_ppl_diff_min:np.float64(-0.004650503396987915) - rollout_corr/ppl_ratio:np.float64(1.000617001671344) - rollout_corr/chi2_token:np.float64(0.0016058182809501886) - rollout_corr/chi2_seq:np.float64(0.5998094449751079) - actor/pg_loss:np.float64(0.005259391153231263) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024053280617408745) - actor/ppo_kl:np.float64(-4.200763549278008e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(142.92776992870495) - self_distillation/token_reweight_w_std:np.float64(3127.1786301988177) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005555916577578) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0809866230993066) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.24522369727492332) - perf/mfu/actor:np.float64(0.05459364258201367) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.3312644958496) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017624415457248688 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.017624415457248688 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:465.7734375 - response_length/max:1081.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.7734375 - response_length_non_aborted/max:1081.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012592412531375885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.609277019277215) - timing_s/agent_loop/generate_sequences/max:np.float64(9.628803690895438) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.290896388272813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.628803690895438) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1081 - timing_s/gen:16.404801338911057 - timing_s/reward:0.06480440311133862 - timing_s/old_log_prob:3.627877188846469 - timing_s/adv:0.5782617889344692 - timing_s/update_actor:21.97222731821239 - timing_s/step:42.739498157054186 - timing_s/stop_profile:0.00011696107685565948 - timing_per_token_ms/update_actor:0.1177415805792298 - timing_per_token_ms/gen:0.13758031281060615 - timing_per_token_ms/adv:0.0030987052897128256 - perf/total_num_tokens:186614 - perf/time_per_step:42.739498157054186 - perf/throughput:545.7890477394363 (TaskRunner pid=2255397) Training Progress: 95%|█████████▌| 95/100 [1:27:59<04:09, 49.88s/it] (TaskRunner pid=2255397) step:96 - global_seqlen/min:20301 - global_seqlen/max:28288 - global_seqlen/minmax_diff:7987 - global_seqlen/balanced_min:23030 - global_seqlen/balanced_max:23033 - global_seqlen/mean:23031.375 - actor/entropy:0.44385668635368347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772989869117737 - training/rollout_probs_diff_mean:0.004754771012812853 - training/rollout_probs_diff_std:0.010701586492359638 - training/rollout_actor_probs_pearson_corr:0.99928879737854 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000027060508728 - rollout_corr/rollout_is_ratio_fraction_high:8.62165597936837e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.084261417388916 - rollout_corr/rollout_is_min:0.5124618411064148 - rollout_corr/rollout_is_std:0.036241523921489716 - rollout_corr/rollout_is_eff_sample_size:0.9986881560335953 - rollout_corr/rollout_is_seq_mean:1.0000017881393433 - rollout_corr/rollout_is_seq_std:0.0018420576816424727 - rollout_corr/rollout_is_seq_max:1.0063093900680542 - rollout_corr/rollout_is_seq_min:0.9944624304771423 - rollout_corr/rollout_is_seq_max_deviation:0.006309390068054199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.598726301919669) - rollout_corr/training_log_ppl:np.float64(0.45704233669675887) - rollout_corr/kl:np.float64(0.0006188384301958338) - rollout_corr/k3_kl:np.float64(0.0007431540022935224) - rollout_corr/rollout_ppl:np.float64(1.5976339941844344) - rollout_corr/rollout_log_ppl:np.float64(0.45642349659465253) - rollout_corr/log_ppl_diff:np.float64(0.0006188401021063328) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015413255896419287) - rollout_corr/log_ppl_diff_max:np.float64(0.0064958930015563965) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055953264236450195) - rollout_corr/ppl_ratio:np.float64(1.0006208247505128) - rollout_corr/chi2_token:np.float64(0.0018268234562128782) - rollout_corr/chi2_seq:np.float64(1.2716746137011796) - actor/pg_loss:np.float64(0.004606917384080589) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021080635951875593) - actor/ppo_kl:np.float64(-7.161125205357166e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(99.39291401347145) - self_distillation/token_reweight_w_std:np.float64(2908.174926977954) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999677997548133) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08357357300701551) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.23777736723423004) - perf/mfu/actor:np.float64(0.05523982050510799) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.2464942932129) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003513324772939086 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003513324772939086 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.07421875 - response_length/max:1135.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.07421875 - response_length_non_aborted/max:1135.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.842030704021454e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1754575334489346) - timing_s/agent_loop/generate_sequences/max:np.float64(9.51023275591433) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9925034317566315) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.51023275591433) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:196 - timing_s/agent_loop/slowest/response_length:1135 - timing_s/gen:16.42838684655726 - timing_s/reward:0.07112030126154423 - timing_s/old_log_prob:3.2094656210392714 - timing_s/adv:0.5936893373727798 - timing_s/update_actor:21.62316256016493 - timing_s/step:42.020951598882675 - timing_s/stop_profile:0.00010878965258598328 - timing_per_token_ms/update_actor:0.11735709743862953 - timing_per_token_ms/gen:0.14163989797612886 - timing_per_token_ms/adv:0.0032221770159878635 - perf/total_num_tokens:184251 - perf/time_per_step:42.020951598882675 - perf/throughput:548.0926567263269 (TaskRunner pid=2255397) Training Progress: 96%|█████████▌| 96/100 [1:28:41<03:10, 47.53s/it] (TaskRunner pid=2255397) step:97 - global_seqlen/min:21184 - global_seqlen/max:28019 - global_seqlen/minmax_diff:6835 - global_seqlen/balanced_min:24523 - global_seqlen/balanced_max:24538 - global_seqlen/mean:24532.875 - actor/entropy:0.43073078989982605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830099582672119 - training/rollout_probs_diff_mean:0.004653396084904671 - training/rollout_probs_diff_std:0.010732254013419151 - training/rollout_actor_probs_pearson_corr:0.9992837905883789 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.999879002571106 - rollout_corr/rollout_is_ratio_fraction_high:8.271640581369866e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.654328116273973e-05 - rollout_corr/rollout_is_max:2.069552421569824 - rollout_corr/rollout_is_min:0.06639717519283295 - rollout_corr/rollout_is_std:0.03590112552046776 - rollout_corr/rollout_is_eff_sample_size:0.9987124115549366 - rollout_corr/rollout_is_seq_mean:0.9998928308486938 - rollout_corr/rollout_is_seq_std:0.001641205046325922 - rollout_corr/rollout_is_seq_max:1.008506178855896 - rollout_corr/rollout_is_seq_min:0.9953399300575256 - rollout_corr/rollout_is_seq_max_deviation:0.008506178855895996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5909629054367542) - rollout_corr/training_log_ppl:np.float64(0.45360937030636705) - rollout_corr/kl:np.float64(0.0007517823871872054) - rollout_corr/k3_kl:np.float64(0.0007286852571155578) - rollout_corr/rollout_ppl:np.float64(1.5898262090049684) - rollout_corr/rollout_log_ppl:np.float64(0.45285758542013355) - rollout_corr/log_ppl_diff:np.float64(0.0007517848862335086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015345667488873005) - rollout_corr/log_ppl_diff_max:np.float64(0.00910806655883789) - rollout_corr/log_ppl_diff_min:np.float64(-0.007346630096435547) - rollout_corr/ppl_ratio:np.float64(1.0007539454381913) - rollout_corr/chi2_token:np.float64(0.00142996059730649) - rollout_corr/chi2_seq:np.float64(0.9373617491219193) - actor/pg_loss:np.float64(0.0030622705817222595) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003035150989489921) - actor/ppo_kl:np.float64(-6.880035599365897e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(482.64709699410014) - self_distillation/token_reweight_w_std:np.float64(7838.689288595924) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0027755422051996) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06985412852372974) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.2353973463177681) - perf/mfu/actor:np.float64(0.058007253459953285) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.27971267700195) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036426237784326077 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036426237784326077 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.24609375 - response_length/max:1302.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.24609375 - response_length_non_aborted/max:1302.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013850070536136627 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9791246633976698) - timing_s/agent_loop/generate_sequences/max:np.float64(10.536064883694053) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.344745161499304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.536064883694053) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:1302 - timing_s/gen:17.246258018538356 - timing_s/reward:0.08357716910541058 - timing_s/old_log_prob:2.9908031839877367 - timing_s/adv:0.5831174645572901 - timing_s/update_actor:22.14829995855689 - timing_s/step:43.14054214023054 - timing_s/stop_profile:0.00011339783668518066 - timing_per_token_ms/update_actor:0.11285010398575834 - timing_per_token_ms/gen:0.14265484940269121 - timing_per_token_ms/adv:0.0029711023705807516 - perf/total_num_tokens:196263 - perf/time_per_step:43.14054214023054 - perf/throughput:568.6733124552454 (TaskRunner pid=2255397) Training Progress: 97%|█████████▋| 97/100 [1:29:24<02:18, 46.23s/it] (TaskRunner pid=2255397) step:98 - global_seqlen/min:21081 - global_seqlen/max:28015 - global_seqlen/minmax_diff:6934 - global_seqlen/balanced_min:24041 - global_seqlen/balanced_max:24240 - global_seqlen/mean:24067.875 - actor/entropy:0.4481050968170166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3950059413909912 - training/rollout_probs_diff_mean:0.0048210457898676395 - training/rollout_probs_diff_std:0.010794337838888168 - training/rollout_actor_probs_pearson_corr:0.9992629289627075 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.000078797340393 - rollout_corr/rollout_is_ratio_fraction_high:3.446166738285683e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7230833691428415e-05 - rollout_corr/rollout_is_max:4.591516971588135 - rollout_corr/rollout_is_min:0.39532703161239624 - rollout_corr/rollout_is_std:0.03653962165117264 - rollout_corr/rollout_is_eff_sample_size:0.9986668739730418 - rollout_corr/rollout_is_seq_mean:1.000028133392334 - rollout_corr/rollout_is_seq_std:0.0019362398888915777 - rollout_corr/rollout_is_seq_max:1.0084097385406494 - rollout_corr/rollout_is_seq_min:0.9942269325256348 - rollout_corr/rollout_is_seq_max_deviation:0.008409738540649414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.616344683803618) - rollout_corr/training_log_ppl:np.float64(0.4696293716551736) - rollout_corr/kl:np.float64(0.0008239057563663721) - rollout_corr/k3_kl:np.float64(0.000780692749458467) - rollout_corr/rollout_ppl:np.float64(1.6150271990336478) - rollout_corr/rollout_log_ppl:np.float64(0.46880546608008444) - rollout_corr/log_ppl_diff:np.float64(0.0008239055750891566) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015701879747211933) - rollout_corr/log_ppl_diff_max:np.float64(0.007200688123703003) - rollout_corr/log_ppl_diff_min:np.float64(-0.004303097724914551) - rollout_corr/ppl_ratio:np.float64(1.0008261122275144) - rollout_corr/chi2_token:np.float64(0.001507394015789032) - rollout_corr/chi2_seq:np.float64(0.24129036953672767) - actor/pg_loss:np.float64(0.003733934834599495) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000350107799476973) - actor/ppo_kl:np.float64(0.00011479198082042785) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(2963.129227875499) - self_distillation/token_reweight_w_std:np.float64(42459.57496579323) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0034989113919437) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09955895229359157) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.2265518605709076) - perf/mfu/actor:np.float64(0.057423452397621935) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.2193260192871) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008424799889326096 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008424799889326096 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.40234375 - response_length/max:1533.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.40234375 - response_length_non_aborted/max:1533.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013044476509094238 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0533475428819656) - timing_s/agent_loop/generate_sequences/max:np.float64(10.802223604172468) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.998830246309808) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.802223604172468) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:188 - timing_s/agent_loop/slowest/response_length:1533 - timing_s/gen:17.642000133171678 - timing_s/reward:0.06738298386335373 - timing_s/old_log_prob:3.014816017821431 - timing_s/adv:0.5671375151723623 - timing_s/update_actor:21.815314499661326 - timing_s/step:43.19491817057133 - timing_s/stop_profile:0.00011375732719898224 - timing_per_token_ms/update_actor:0.11330100029427882 - timing_per_token_ms/gen:0.15199317773751994 - timing_per_token_ms/adv:0.002945510951695789 - perf/total_num_tokens:192543 - perf/time_per_step:43.19491817057133 - perf/throughput:557.1922813919678 (TaskRunner pid=2255397) Training Progress: 98%|█████████▊| 98/100 [1:30:08<01:30, 45.33s/it] (TaskRunner pid=2255397) step:99 - global_seqlen/min:19692 - global_seqlen/max:25383 - global_seqlen/minmax_diff:5691 - global_seqlen/balanced_min:23698 - global_seqlen/balanced_max:23716 - global_seqlen/mean:23712.375 - actor/entropy:0.4742501676082611 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4091079831123352 - training/rollout_probs_diff_mean:0.004931129980832338 - training/rollout_probs_diff_std:0.010619550943374634 - training/rollout_actor_probs_pearson_corr:0.9992989301681519 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.9998912215232849 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.113195119425654e-06 - rollout_corr/rollout_is_max:1.7866209745407104 - rollout_corr/rollout_is_min:0.43442291021347046 - rollout_corr/rollout_is_std:0.03628097102046013 - rollout_corr/rollout_is_eff_sample_size:0.9986851836242739 - rollout_corr/rollout_is_seq_mean:0.9998824596405029 - rollout_corr/rollout_is_seq_std:0.00225405883975327 - rollout_corr/rollout_is_seq_max:1.0066941976547241 - rollout_corr/rollout_is_seq_min:0.9925881028175354 - rollout_corr/rollout_is_seq_max_deviation:0.0074118971824646 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6540217595174909) - rollout_corr/training_log_ppl:np.float64(0.49091246334137395) - rollout_corr/kl:np.float64(0.0009071645049343324) - rollout_corr/k3_kl:np.float64(0.0007826964022683569) - rollout_corr/rollout_ppl:np.float64(1.6525658327154815) - rollout_corr/rollout_log_ppl:np.float64(0.4900052981101908) - rollout_corr/log_ppl_diff:np.float64(0.0009071652311831713) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018514589173719287) - rollout_corr/log_ppl_diff_max:np.float64(0.008251607418060303) - rollout_corr/log_ppl_diff_min:np.float64(-0.005766570568084717) - rollout_corr/ppl_ratio:np.float64(1.0009100551251322) - rollout_corr/chi2_token:np.float64(0.001357766566798091) - rollout_corr/chi2_seq:np.float64(1.0302318546455353) - actor/pg_loss:np.float64(0.004117259988561273) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030947469667808036) - actor/ppo_kl:np.float64(9.188319657127408e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(190.36794193601236) - self_distillation/token_reweight_w_std:np.float64(4398.700665255601) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010327426716685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09465871116844937) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.2390700727701187) - perf/mfu/actor:np.float64(0.055725058096870825) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.29999542236328) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006170772016048431 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006170772016048431 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.63671875 - response_length/max:1227.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.63671875 - response_length_non_aborted/max:1227.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001377556473016739 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2593938652426004) - timing_s/agent_loop/generate_sequences/max:np.float64(10.128664135932922) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.985631817675312) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.128664135932922) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:1227 - timing_s/gen:17.193039959296584 - timing_s/reward:0.06556063517928123 - timing_s/old_log_prob:2.973387535661459 - timing_s/adv:0.5816671065986156 - timing_s/update_actor:21.934669038280845 - timing_s/step:42.834324315190315 - timing_s/stop_profile:0.00016217492520809174 - timing_per_token_ms/update_actor:0.11562880689028854 - timing_per_token_ms/gen:0.15668352570646932 - timing_per_token_ms/adv:0.003066263431007099 - perf/total_num_tokens:189699 - perf/time_per_step:42.834324315190315 - perf/throughput:553.5834959252735 (TaskRunner pid=2255397) Training Progress: 99%|█████████▉| 99/100 [1:30:51<00:44, 44.60s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure of the product and compare it with the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains: (TaskRunner pid=2255397) - A **nitrogen-containing ring** (likely an imine or azole ring), (TaskRunner pid=2255397) - A **chlorine atom** attached to a benzene ring, (TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F), (TaskRunner pid=2255397) - A **nitrogen atom** connected to a carbon chain containing a **chlorinated benzene ring** and a **fluorinated group**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let’s examine the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A:** (TaskRunner pid=2255397) - Contains a **nitrogen-containing ring** (likely a pyrrole or similar), (TaskRunner pid=2255397) - A **chlorinated benzene ring** (with Cl), (TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F), (TaskRunner pid=2255397) - A **nitrogen atom** connected to a carbon chain that includes the **chlorinated benzene** and **fluorinated group**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) This matches the structure of the target product very closely, including all the key functional groups and their positions. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B:** (TaskRunner pid=2255397) - Contains a **chlorine** and **nitrogen**, but the **fluorinated group** is not present in the correct position, and the **nitrogen ring** is not properly structured. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C:** (TaskRunner pid=2255397) - Contains a **fluorinated group**, but the **chlorine** is attached to an **oxygen**, not directly to the benzene ring. The **fluorinated group** is in a different position than in the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D:** (TaskRunner pid=2255397) - Contains a **fluorinated group** and a **chlorinated benzene ring**, but the **nitrogen-containing ring** is not present in the correct structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Only **Option A** contains all the required functional groups in the correct positions to serve as the **only correct reactant** for the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2255397) step:100 - global_seqlen/min:23741 - global_seqlen/max:31057 - global_seqlen/minmax_diff:7316 - global_seqlen/balanced_min:24739 - global_seqlen/balanced_max:31757 - global_seqlen/mean:25617.625 - actor/entropy:0.4436495304107666 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24008721113204956 - training/rollout_probs_diff_mean:0.004401609301567078 - training/rollout_probs_diff_std:0.009758861735463142 - training/rollout_actor_probs_pearson_corr:0.9993872046470642 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.91796875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.91796875 - rollout_corr/rollout_is_mean:1.0000593662261963 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7518070936203003 - rollout_corr/rollout_is_min:0.5589723587036133 - rollout_corr/rollout_is_std:0.03378680348396301 - rollout_corr/rollout_is_eff_sample_size:0.9988599913647978 - rollout_corr/rollout_is_seq_mean:1.0000724792480469 - rollout_corr/rollout_is_seq_std:0.0018252491718158126 - rollout_corr/rollout_is_seq_max:1.0057400465011597 - rollout_corr/rollout_is_seq_min:0.9935486912727356 - rollout_corr/rollout_is_seq_max_deviation:0.006451308727264404 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6480820523574948) - rollout_corr/training_log_ppl:np.float64(0.4857633382116546) - rollout_corr/kl:np.float64(0.0007351505751955756) - rollout_corr/k3_kl:np.float64(0.0006855184932135217) - rollout_corr/rollout_ppl:np.float64(1.6468097120523453) - rollout_corr/rollout_log_ppl:np.float64(0.48502818830229444) - rollout_corr/log_ppl_diff:np.float64(0.0007351499093601888) - rollout_corr/log_ppl_abs_diff:np.float64(0.001427958528893214) - rollout_corr/log_ppl_diff_max:np.float64(0.007762610912322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.004005610942840576) - rollout_corr/ppl_ratio:np.float64(1.0007370123639703) - rollout_corr/chi2_token:np.float64(0.0012815699446946383) - rollout_corr/chi2_seq:np.float64(0.31913869245909154) - actor/pg_loss:np.float64(0.003367550321854651) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003017752374034899) - actor/ppo_kl:np.float64(0.00018170115079385596) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.91796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.91796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(1570.6249493323267) - self_distillation/token_reweight_w_std:np.float64(40484.064735737746) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001023317920044) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0961508322507143) - self_distillation/empty_target_batch:np.float64(0.08203125) - actor/grad_norm:np.float64(0.2478880025446415) - perf/mfu/actor:np.float64(0.05716674172282474) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.12733459472656) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7065476190476191) - val-aux/sciknoweval/reward/std@16:np.float64(0.14963967195739616) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7672380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12522806043858783) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6460952380952382) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12793485078945213) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7068333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14978365629442147) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8158333333333333) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09352166658220648) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5964190476190476) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09674390853009297) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7151904761904762) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1197973344827459) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8520904761904763) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06467821428517558) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5596142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06871965760347537) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7202857142857142) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08852864730736695) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8776952380952381) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0403808452308748) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5314904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04527152807209723) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7233190476190475) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06487826170016521) - val-aux/sciknoweval/score/mean@16:np.float64(0.7065476190476191) - val-aux/sciknoweval/score/std@16:np.float64(0.14963967195739616) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7672380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12522806043858783) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6460952380952382) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.12793485078945213) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7068333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14978365629442147) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8158333333333333) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09352166658220648) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5964190476190476) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09674390853009297) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7151904761904762) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1197973344827459) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8520904761904763) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06467821428517558) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5596142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06871965760347537) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7202857142857142) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08852864730736695) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8776952380952381) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0403808452308748) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5314904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04527152807209723) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7233190476190475) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06487826170016521) - val-core/sciknoweval/acc/mean@16:np.float64(0.7065476190476191) - val-aux/sciknoweval/acc/std@16:np.float64(0.14963967195739616) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7672380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12522806043858783) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6460952380952382) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.12793485078945213) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7068333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14978365629442147) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8158333333333333) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09352166658220648) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5964190476190476) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09674390853009297) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7151904761904762) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1197973344827459) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8520904761904763) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06467821428517558) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5596142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06871965760347537) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7202857142857142) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08852864730736695) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8776952380952381) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0403808452308748) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5314904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04527152807209723) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7233190476190475) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06487826170016521) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9982142857142857) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.006916041689656103) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.999904761904762) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0016180983939162673) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9966190476190476) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.009226171602153282) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9982380952380953) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.006862312067166463) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9934619047619048) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.012001753208129275) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9993142857142857) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0043365672892029085) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.988647619047619) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.01397823789672266) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9998380952380953) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0019314678388328032) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9817619047619048) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.013705261025670075) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006015850374731164) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026495642960071564 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026495642960071564 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:504.39453125 - response_length/max:8192.0 - response_length/min:195.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:504.39453125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013913773000240326 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3870061337947845) - timing_s/agent_loop/generate_sequences/max:np.float64(60.303343798965216) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.73472642654815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.303343798965216) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.0998271573335 - timing_s/reward:0.0683623980730772 - timing_s/old_log_prob:3.3458864334970713 - timing_s/adv:0.5801629964262247 - timing_s/update_actor:23.488009363412857 - timing_s/step:94.66876287944615 - timing_s/testing:143.36135754734278 - timing_s/save_checkpoint:13.737380957230926 - timing_s/stop_profile:0.00018646568059921265 - timing_per_token_ms/update_actor:0.11460864035704353 - timing_per_token_ms/gen:0.5196501619154579 - timing_per_token_ms/adv:0.002830878137738299 - perf/total_num_tokens:204941 - perf/time_per_step:94.66876287944615 - perf/throughput:270.6027228075453 (TaskRunner pid=2255397) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2255397) "np.float64(0.7065476190476191), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2255397) "np.float64(0.14963967195739616), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7672380952380953), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2255397) "np.float64(0.12522806043858783), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2255397) "np.float64(0.6460952380952382), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2255397) "np.float64(0.12793485078945213), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7068333333333333), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2255397) "np.float64(0.14978365629442147), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2255397) "np.float64(0.8158333333333333), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2255397) "np.float64(0.09352166658220648), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2255397) "np.float64(0.5964190476190476), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2255397) "np.float64(0.09674390853009297), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2255397) "np.float64(0.7151904761904762), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2255397) "np.float64(0.1197973344827459), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2255397) "np.float64(0.8520904761904763), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2255397) "np.float64(0.06467821428517558), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2255397) "np.float64(0.5596142857142857), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2255397) "np.float64(0.06871965760347537), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2255397) "np.float64(0.7202857142857142), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2255397) "np.float64(0.08852864730736695), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2255397) "np.float64(0.8776952380952381), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2255397) "np.float64(0.0403808452308748), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2255397) "np.float64(0.5314904761904762), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2255397) "np.float64(0.04527152807209723), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2255397) "np.float64(0.7233190476190475), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2255397) "np.float64(0.06487826170016521), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2255397) "np.float64(0.7065476190476191), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2255397) "np.float64(0.14963967195739616), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7672380952380953), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2255397) "np.float64(0.12522806043858783), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2255397) "np.float64(0.6460952380952382), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2255397) "np.float64(0.12793485078945213), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7068333333333333), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2255397) "np.float64(0.14978365629442147), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2255397) "np.float64(0.8158333333333333), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2255397) "np.float64(0.09352166658220648), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2255397) "np.float64(0.5964190476190476), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2255397) "np.float64(0.09674390853009297), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2255397) "np.float64(0.7151904761904762), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2255397) "np.float64(0.1197973344827459), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2255397) "np.float64(0.8520904761904763), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2255397) "np.float64(0.06467821428517558), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2255397) "np.float64(0.5596142857142857), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2255397) "np.float64(0.06871965760347537), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2255397) "np.float64(0.7202857142857142), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2255397) "np.float64(0.08852864730736695), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2255397) "np.float64(0.8776952380952381), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2255397) "np.float64(0.0403808452308748), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2255397) "np.float64(0.5314904761904762), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2255397) "np.float64(0.04527152807209723), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2255397) "np.float64(0.7233190476190475), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2255397) "np.float64(0.06487826170016521), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2255397) "np.float64(0.7065476190476191), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2255397) "np.float64(0.14963967195739616), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7672380952380953), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2255397) "np.float64(0.12522806043858783), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2255397) "np.float64(0.6460952380952382), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2255397) "np.float64(0.12793485078945213), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7068333333333333), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2255397) "np.float64(0.14978365629442147), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2255397) "np.float64(0.8158333333333333), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2255397) "np.float64(0.09352166658220648), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2255397) "np.float64(0.5964190476190476), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2255397) "np.float64(0.09674390853009297), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2255397) "np.float64(0.7151904761904762), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2255397) "np.float64(0.1197973344827459), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2255397) "np.float64(0.8520904761904763), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2255397) "np.float64(0.06467821428517558), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2255397) "np.float64(0.5596142857142857), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2255397) "np.float64(0.06871965760347537), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2255397) "np.float64(0.7202857142857142), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2255397) "np.float64(0.08852864730736695), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2255397) "np.float64(0.8776952380952381), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2255397) "np.float64(0.0403808452308748), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2255397) "np.float64(0.5314904761904762), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2255397) "np.float64(0.04527152807209723), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2255397) "np.float64(0.7233190476190475), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2255397) 'np.float64(0.06487826170016521), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2255397) 'np.float64(0.9982142857142857), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2255397) 'np.float64(0.006916041689656103), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2255397) 'np.float64(0.999904761904762), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2255397) 'np.float64(0.0016180983939162673), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2255397) 'np.float64(0.9966190476190476), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2255397) 'np.float64(0.009226171602153282), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': " (TaskRunner pid=2255397) 'np.float64(0.9982380952380953), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2255397) 'np.float64(0.006862312067166463), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2255397) 'np.float64(0.9934619047619048), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2255397) 'np.float64(0.012001753208129275), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2255397) 'np.float64(0.9993142857142857), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2255397) 'np.float64(0.0043365672892029085), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2255397) 'np.float64(0.988647619047619), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2255397) 'np.float64(0.01397823789672266), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2255397) 'np.float64(0.9998380952380953), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2255397) 'np.float64(0.0019314678388328032), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2255397) 'np.float64(0.9817619047619048), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2255397) 'np.float64(0.013705261025670075), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2255397) 'np.float64(0.9999714285714286), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2255397) "np.float64(0.0006015850374731164), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2255397) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2255397) 'np.float64(2.0)}') (TaskRunner pid=2255397) Training Progress: 100%|██████████| 100/100 [1:35:02<00:00, 106.78s/it] Training Progress: 100%|██████████| 100/100 [1:35:02<00:00, 57.03s/it] (TaskRunner pid=2255397) Traceback (most recent call last): (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2255397) worker.main_loop() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2255397) self.core_worker.run_task_loop() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2255397) _error_if_any_worker_fails() (TaskRunner pid=2255397) RuntimeError: DataLoader worker (pid 2270092) is killed by signal: Killed. (TaskRunner pid=2255397) Traceback (most recent call last): (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2255397) worker.main_loop() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2255397) self.core_worker.run_task_loop() (TaskRunner pid=2255397) Exception ignored in: (TaskRunner pid=2255397) Traceback (most recent call last): (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/verl/utils/tracking.py", line 169, in __del__ (TaskRunner pid=2255397) self.logger["wandb"].finish(exit_code=0) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 4097, in finish (TaskRunner pid=2255397) wandb.run.finish(exit_code=exit_code, quiet=quiet) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2255397) return func(self, *args, **kwargs) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 445, in wrapper (TaskRunner pid=2255397) return func(self, *args, **kwargs) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2275, in finish (TaskRunner pid=2255397) return self._finish(exit_code) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2255397) return func(self, *args, **kwargs) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2288, in _finish (TaskRunner pid=2255397) with telemetry.context(run=self) as tel: (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/telemetry.py", line 42, in __exit__ (TaskRunner pid=2255397) self._run._telemetry_callback(self._obj) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 752, in _telemetry_callback (TaskRunner pid=2255397) self._telemetry_flush() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 765, in _telemetry_flush (TaskRunner pid=2255397) self._backend.interface._publish_telemetry(self._telemetry_obj) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_shared.py", line 103, in _publish_telemetry (TaskRunner pid=2255397) self._publish(rec) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 46, in _publish (TaskRunner pid=2255397) self._asyncer.run(lambda: self._client.publish(request)) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2255397) return future.result() (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2255397) return self.__get_result() (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2255397) raise self._exception (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2255397) return await fn() (TaskRunner pid=2255397) ^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 38, in publish (TaskRunner pid=2255397) await self._send_server_request(request) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 59, in _send_server_request (TaskRunner pid=2255397) self._writer.write(header) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 346, in write (TaskRunner pid=2255397) self._transport.write(data) (TaskRunner pid=2255397) File "uvloop/handles/stream.pyx", line 675, in uvloop.loop.UVStream.write (TaskRunner pid=2255397) File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive (TaskRunner pid=2255397) RuntimeError: unable to perform operation on ; the handler is closed main_ppo exit code: 0 [2026-07-03 03:18:43] Finished Qwen/Qwen3-8B chemistry rlsd_tr. [2026-07-03 03:18:43] Starting Qwen/Qwen3-8B physics grpo. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_8B 2026-07-03 03:18:44,953 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 03:18:54,758 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2281424) TaskRunner hostname: mole-workspace-rw, PID: 2281424 (TaskRunner pid=2281424) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2281424) 'calculate_entropy': False, (TaskRunner pid=2281424) 'calculate_sum_pi_squared': False, (TaskRunner pid=2281424) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2281424) 'async_save': False, (TaskRunner pid=2281424) 'load_contents': ['model', (TaskRunner pid=2281424) 'optimizer', (TaskRunner pid=2281424) 'extra'], (TaskRunner pid=2281424) 'save_contents': ['model', (TaskRunner pid=2281424) 'optimizer', (TaskRunner pid=2281424) 'extra']}, (TaskRunner pid=2281424) 'clip_ratio': 0.2, (TaskRunner pid=2281424) 'clip_ratio_c': 3.0, (TaskRunner pid=2281424) 'clip_ratio_high': 0.28, (TaskRunner pid=2281424) 'clip_ratio_low': 0.2, (TaskRunner pid=2281424) 'data_loader_seed': 42, (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_coeff': 0, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'freeze_vision_tower': False, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'forward_only': False, (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'full_determinism': False, (TaskRunner pid=2281424) 'model_dtype': 'fp32', (TaskRunner pid=2281424) 'offload_policy': False, (TaskRunner pid=2281424) 'optimizer_offload': False, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'seed': 42, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_orig_params': False, (TaskRunner pid=2281424) 'use_torch_compile': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'grad_clip': 1.0, (TaskRunner pid=2281424) 'kl_loss_coef': 0.0, (TaskRunner pid=2281424) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2281424) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2281424) 'loss_scale_factor': None, (TaskRunner pid=2281424) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2281424) 'betas': [0.9, 0.999], (TaskRunner pid=2281424) 'clip_grad': 1.0, (TaskRunner pid=2281424) 'lr': 1e-06, (TaskRunner pid=2281424) 'lr_scheduler_type': 'constant', (TaskRunner pid=2281424) 'lr_warmup_steps': 10, (TaskRunner pid=2281424) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2281424) 'min_lr_ratio': 0.0, (TaskRunner pid=2281424) 'num_cycles': 0.5, (TaskRunner pid=2281424) 'optimizer': 'AdamW', (TaskRunner pid=2281424) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2281424) 'override_optimizer_config': None, (TaskRunner pid=2281424) 'total_training_steps': -1, (TaskRunner pid=2281424) 'warmup_style': None, (TaskRunner pid=2281424) 'weight_decay': 0.01}, (TaskRunner pid=2281424) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2281424) 'clip_cov_lb': 1.0, (TaskRunner pid=2281424) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2281424) 'clip_cov_ub': 5.0, (TaskRunner pid=2281424) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2281424) 'loss_mode': 'vanilla', (TaskRunner pid=2281424) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2281424) 'ppo_epochs': 1, (TaskRunner pid=2281424) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2281424) 'ppo_micro_batch_size': None, (TaskRunner pid=2281424) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2281424) 'ppo_mini_batch_size': 8, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'rollout_n': 8, (TaskRunner pid=2281424) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2281424) 'mode': 'disabled', (TaskRunner pid=2281424) 'record_file': None, (TaskRunner pid=2281424) 'replay_file': None}, (TaskRunner pid=2281424) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2281424) 'alpha': 0.5, (TaskRunner pid=2281424) 'distillation_add_tail': True, (TaskRunner pid=2281424) 'distillation_topk': 100, (TaskRunner pid=2281424) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2281424) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2281424) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'loss_weight': 0.0, (TaskRunner pid=2281424) 'mask': 'all'}, (TaskRunner pid=2281424) 'feedback_template': '\n' (TaskRunner pid=2281424) 'The ' (TaskRunner pid=2281424) 'following ' (TaskRunner pid=2281424) 'is ' (TaskRunner pid=2281424) 'feedback ' (TaskRunner pid=2281424) 'from ' (TaskRunner pid=2281424) 'your ' (TaskRunner pid=2281424) 'unsuccessful ' (TaskRunner pid=2281424) 'earlier ' (TaskRunner pid=2281424) 'attempt:\n' (TaskRunner pid=2281424) '\n' (TaskRunner pid=2281424) '{feedback_raw}', (TaskRunner pid=2281424) 'full_logit_distillation': True, (TaskRunner pid=2281424) 'include_environment_feedback': True, (TaskRunner pid=2281424) 'is_clip': 2, (TaskRunner pid=2281424) 'max_reprompt_len': 22528, (TaskRunner pid=2281424) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2281424) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2281424) '\n' (TaskRunner pid=2281424) 'Correctly ' (TaskRunner pid=2281424) 'solve ' (TaskRunner pid=2281424) 'the ' (TaskRunner pid=2281424) 'original ' (TaskRunner pid=2281424) 'question.', (TaskRunner pid=2281424) 'reprompt_truncation': 'right', (TaskRunner pid=2281424) 'solution_template': '\n' (TaskRunner pid=2281424) 'Correct ' (TaskRunner pid=2281424) 'solution:\n' (TaskRunner pid=2281424) '\n' (TaskRunner pid=2281424) '{successful_previous_attempt}', (TaskRunner pid=2281424) 'srpo_dynamic_weighting': False, (TaskRunner pid=2281424) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2281424) 'success_reward_threshold': 0.5, (TaskRunner pid=2281424) 'teacher_regularization': 'ema', (TaskRunner pid=2281424) 'teacher_update_rate': 0.0, (TaskRunner pid=2281424) 'token_reweight_decay_steps': None, (TaskRunner pid=2281424) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2281424) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2281424) 'shuffle': False, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2281424) 'tau_neg': 1.05, (TaskRunner pid=2281424) 'tau_pos': 1.0, (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_dynamic_bsz': False, (TaskRunner pid=2281424) 'use_fused_kernels': False, (TaskRunner pid=2281424) 'use_kl_loss': False, (TaskRunner pid=2281424) 'use_prefix_grouper': False, (TaskRunner pid=2281424) 'use_remove_padding': True, (TaskRunner pid=2281424) 'use_torch_compile': True}, (TaskRunner pid=2281424) 'hybrid_engine': True, (TaskRunner pid=2281424) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2281424) 'custom_chat_template': None, (TaskRunner pid=2281424) 'enable_activation_offload': False, (TaskRunner pid=2281424) 'enable_gradient_checkpointing': True, (TaskRunner pid=2281424) 'exclude_modules': None, (TaskRunner pid=2281424) 'external_lib': None, (TaskRunner pid=2281424) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2281424) 'hf_config_path': None, (TaskRunner pid=2281424) 'lora_adapter_path': None, (TaskRunner pid=2281424) 'lora_alpha': 16, (TaskRunner pid=2281424) 'lora_rank': 0, (TaskRunner pid=2281424) 'override_config': {}, (TaskRunner pid=2281424) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'target_modules': 'all-linear', (TaskRunner pid=2281424) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2281424) 'num_shards': 4}, (TaskRunner pid=2281424) 'tokenizer_path': None, (TaskRunner pid=2281424) 'trust_remote_code': True, (TaskRunner pid=2281424) 'use_fused_kernels': False, (TaskRunner pid=2281424) 'use_liger': False, (TaskRunner pid=2281424) 'use_remove_padding': True, (TaskRunner pid=2281424) 'use_shm': False}, (TaskRunner pid=2281424) 'nccl_timeout': 600, (TaskRunner pid=2281424) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'forward_only': True, (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'full_determinism': False, (TaskRunner pid=2281424) 'model_dtype': 'fp32', (TaskRunner pid=2281424) 'offload_policy': False, (TaskRunner pid=2281424) 'optimizer_offload': False, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'seed': 42, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_orig_params': False, (TaskRunner pid=2281424) 'use_torch_compile': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2281424) 'log_prob_micro_batch_size': None, (TaskRunner pid=2281424) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2281424) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'rollout_n': 8, (TaskRunner pid=2281424) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2281424) 'mode': 'disabled', (TaskRunner pid=2281424) 'record_file': None, (TaskRunner pid=2281424) 'replay_file': None}, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_torch_compile': True}, (TaskRunner pid=2281424) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2281424) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2281424) 'agent_loop_config_path': None, (TaskRunner pid=2281424) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2281424) 'name': None, (TaskRunner pid=2281424) 'path': None}, (TaskRunner pid=2281424) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2281424) 'num_workers': 8}, (TaskRunner pid=2281424) 'calculate_log_probs': True, (TaskRunner pid=2281424) 'cudagraph_capture_sizes': None, (TaskRunner pid=2281424) 'data_parallel_size': 1, (TaskRunner pid=2281424) 'disable_log_stats': True, (TaskRunner pid=2281424) 'do_sample': True, (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'enable_chunked_prefill': True, (TaskRunner pid=2281424) 'enable_prefix_caching': True, (TaskRunner pid=2281424) 'enable_rollout_routing_replay': False, (TaskRunner pid=2281424) 'enforce_eager': False, (TaskRunner pid=2281424) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2281424) 'expert_parallel_size': 1, (TaskRunner pid=2281424) 'free_cache_engine': True, (TaskRunner pid=2281424) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2281424) 'ignore_eos': False, (TaskRunner pid=2281424) 'layered_summon': False, (TaskRunner pid=2281424) 'load_format': 'dummy', (TaskRunner pid=2281424) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2281424) 'log_prob_micro_batch_size': None, (TaskRunner pid=2281424) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2281424) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2281424) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2281424) 'max_model_len': 10240, (TaskRunner pid=2281424) 'max_num_batched_tokens': 10240, (TaskRunner pid=2281424) 'max_num_seqs': 1024, (TaskRunner pid=2281424) 'mode': 'async', (TaskRunner pid=2281424) 'multi_stage_wake_up': False, (TaskRunner pid=2281424) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'format': 'hermes', (TaskRunner pid=2281424) 'interaction_config_path': None, (TaskRunner pid=2281424) 'max_assistant_turns': None, (TaskRunner pid=2281424) 'max_parallel_calls': 1, (TaskRunner pid=2281424) 'max_tool_response_length': 256, (TaskRunner pid=2281424) 'max_user_turns': None, (TaskRunner pid=2281424) 'num_repeat_rollouts': None, (TaskRunner pid=2281424) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2281424) 'tool_config_path': None, (TaskRunner pid=2281424) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2281424) 'use_inference_chat_template': False}, (TaskRunner pid=2281424) 'n': 8, (TaskRunner pid=2281424) 'name': 'vllm', (TaskRunner pid=2281424) 'over_sample_rate': 0, (TaskRunner pid=2281424) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2281424) 'port': 9090, (TaskRunner pid=2281424) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2281424) 'prompt_length': 2048, (TaskRunner pid=2281424) 'quantization': None, (TaskRunner pid=2281424) 'quantization_config_file': None, (TaskRunner pid=2281424) 'response_length': 8192, (TaskRunner pid=2281424) 'scheduling_policy': 'fcfs', (TaskRunner pid=2281424) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2281424) 'skip_rollout': False, (TaskRunner pid=2281424) 'skip_tokenizer_init': True, (TaskRunner pid=2281424) 'temperature': 1.0, (TaskRunner pid=2281424) 'tensor_model_parallel_size': 2, (TaskRunner pid=2281424) 'top_k': -1, (TaskRunner pid=2281424) 'top_p': 1.0, (TaskRunner pid=2281424) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2281424) 'backend': None, (TaskRunner pid=2281424) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2281424) 'token2text': False}, (TaskRunner pid=2281424) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2281424) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2281424) 'do_sample': True, (TaskRunner pid=2281424) 'n': 16, (TaskRunner pid=2281424) 'temperature': 0.6, (TaskRunner pid=2281424) 'top_k': -1, (TaskRunner pid=2281424) 'top_p': 0.95}}}, (TaskRunner pid=2281424) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2281424) 'adv_estimator': 'grpo', (TaskRunner pid=2281424) 'gamma': 1.0, (TaskRunner pid=2281424) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2281424) 'horizon': 10000, (TaskRunner pid=2281424) 'kl_coef': 0.001, (TaskRunner pid=2281424) 'target_kl': 0.1, (TaskRunner pid=2281424) 'type': 'fixed'}, (TaskRunner pid=2281424) 'kl_penalty': 'kl', (TaskRunner pid=2281424) 'lam': 1.0, (TaskRunner pid=2281424) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2281424) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2281424) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2281424) 'loss_type': 'ppo_clip', (TaskRunner pid=2281424) 'rollout_is': 'token', (TaskRunner pid=2281424) 'rollout_is_batch_normalize': False, (TaskRunner pid=2281424) 'rollout_is_threshold': 2.0, (TaskRunner pid=2281424) 'rollout_rs': None, (TaskRunner pid=2281424) 'rollout_rs_threshold': None}, (TaskRunner pid=2281424) 'use_kl_in_reward': False, (TaskRunner pid=2281424) 'use_pf_ppo': False}, (TaskRunner pid=2281424) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2281424) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2281424) 'async_save': False, (TaskRunner pid=2281424) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2281424) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2281424) 'cliprange_value': 0.5, (TaskRunner pid=2281424) 'data_loader_seed': 42, (TaskRunner pid=2281424) 'enable': None, (TaskRunner pid=2281424) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2281424) 'forward_micro_batch_size': None, (TaskRunner pid=2281424) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2281424) 'grad_clip': 1.0, (TaskRunner pid=2281424) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2281424) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2281424) 'enable_activation_offload': False, (TaskRunner pid=2281424) 'enable_gradient_checkpointing': True, (TaskRunner pid=2281424) 'external_lib': None, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'forward_only': False, (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'full_determinism': False, (TaskRunner pid=2281424) 'model_dtype': 'fp32', (TaskRunner pid=2281424) 'offload_policy': False, (TaskRunner pid=2281424) 'optimizer_offload': False, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'seed': 42, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_orig_params': False, (TaskRunner pid=2281424) 'use_torch_compile': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'lora_alpha': 16, (TaskRunner pid=2281424) 'lora_rank': 0, (TaskRunner pid=2281424) 'override_config': {}, (TaskRunner pid=2281424) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'target_modules': 'all-linear', (TaskRunner pid=2281424) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2281424) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'trust_remote_code': True, (TaskRunner pid=2281424) 'use_remove_padding': False, (TaskRunner pid=2281424) 'use_shm': False}, (TaskRunner pid=2281424) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2281424) 'betas': [0.9, 0.999], (TaskRunner pid=2281424) 'clip_grad': 1.0, (TaskRunner pid=2281424) 'lr': 1e-05, (TaskRunner pid=2281424) 'lr_scheduler_type': 'constant', (TaskRunner pid=2281424) 'lr_warmup_steps': -1, (TaskRunner pid=2281424) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2281424) 'min_lr_ratio': 0.0, (TaskRunner pid=2281424) 'num_cycles': 0.5, (TaskRunner pid=2281424) 'optimizer': 'AdamW', (TaskRunner pid=2281424) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2281424) 'override_optimizer_config': None, (TaskRunner pid=2281424) 'total_training_steps': -1, (TaskRunner pid=2281424) 'warmup_style': None, (TaskRunner pid=2281424) 'weight_decay': 0.01}, (TaskRunner pid=2281424) 'ppo_epochs': 1, (TaskRunner pid=2281424) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2281424) 'ppo_micro_batch_size': None, (TaskRunner pid=2281424) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2281424) 'ppo_mini_batch_size': 8, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'rollout_n': 8, (TaskRunner pid=2281424) 'shuffle': False, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_dynamic_bsz': False}, (TaskRunner pid=2281424) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2281424) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2281424) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2281424) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2281424) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2281424) 'dataloader_num_workers': 8, (TaskRunner pid=2281424) 'filter_overlong_prompts': True, (TaskRunner pid=2281424) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2281424) 'image_key': 'images', (TaskRunner pid=2281424) 'image_patch_size': 14, (TaskRunner pid=2281424) 'max_prompt_length': 2048, (TaskRunner pid=2281424) 'max_response_length': 8192, (TaskRunner pid=2281424) 'prompt_key': 'prompt', (TaskRunner pid=2281424) 'return_full_prompt': False, (TaskRunner pid=2281424) 'return_multi_modal_inputs': True, (TaskRunner pid=2281424) 'return_raw_chat': True, (TaskRunner pid=2281424) 'return_raw_input_ids': False, (TaskRunner pid=2281424) 'reward_fn_key': 'data_source', (TaskRunner pid=2281424) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2281424) 'seed': None, (TaskRunner pid=2281424) 'shuffle': True, (TaskRunner pid=2281424) 'tokenizer': None, (TaskRunner pid=2281424) 'tool_config_path': None, (TaskRunner pid=2281424) 'train_batch_size': 32, (TaskRunner pid=2281424) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2281424) 'train_max_samples': 3200, (TaskRunner pid=2281424) 'truncation': 'error', (TaskRunner pid=2281424) 'trust_remote_code': True, (TaskRunner pid=2281424) 'use_shm': False, (TaskRunner pid=2281424) 'val_batch_size': None, (TaskRunner pid=2281424) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2281424) 'val_max_samples': -1, (TaskRunner pid=2281424) 'validation_shuffle': False, (TaskRunner pid=2281424) 'video_key': 'videos'}, (TaskRunner pid=2281424) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2281424) 'cuda-memory-usage': 'true', (TaskRunner pid=2281424) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2281424) 'capture-range-end': None, (TaskRunner pid=2281424) 'cuda-graph-trace': 'graph', (TaskRunner pid=2281424) 'cuda-memory-usage': 'true', (TaskRunner pid=2281424) 'kill': 'none', (TaskRunner pid=2281424) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2281424) 'torch_memory': {'context': 'all', (TaskRunner pid=2281424) 'kw_args': {}, (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'stacks': 'all', (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2281424) 'profile_continuous_steps': False, (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'steps': None, (TaskRunner pid=2281424) 'tool': None}, (TaskRunner pid=2281424) 'max_model_len': 10240, (TaskRunner pid=2281424) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_8B', (TaskRunner pid=2281424) 'include_dashboard': False, (TaskRunner pid=2281424) 'num_cpus': None, (TaskRunner pid=2281424) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2281424) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2281424) 'USER': 'root'}}}, (TaskRunner pid=2281424) 'timeline_json_file': None}, (TaskRunner pid=2281424) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2281424) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2281424) 'name': 'custom_reward_manager', (TaskRunner pid=2281424) 'path': None}, (TaskRunner pid=2281424) 'name': 'naive', (TaskRunner pid=2281424) 'source': 'register'}, (TaskRunner pid=2281424) 'reward_model': {'enable': False, (TaskRunner pid=2281424) 'enable_resource_pool': False, (TaskRunner pid=2281424) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2281424) 'launch_reward_fn_async': False, (TaskRunner pid=2281424) 'max_length': None, (TaskRunner pid=2281424) 'micro_batch_size': None, (TaskRunner pid=2281424) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2281424) 'model': {'external_lib': None, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'override_config': {}, (TaskRunner pid=2281424) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2281424) 'trust_remote_code': False, (TaskRunner pid=2281424) 'use_fused_kernels': False, (TaskRunner pid=2281424) 'use_remove_padding': False, (TaskRunner pid=2281424) 'use_shm': False}, (TaskRunner pid=2281424) 'n_gpus_per_node': 8, (TaskRunner pid=2281424) 'nnodes': 0, (TaskRunner pid=2281424) 'num_workers': 1, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'reward_loop_class_name': None, (TaskRunner pid=2281424) 'reward_loop_module_path': None, (TaskRunner pid=2281424) 'reward_loop_source': 'register', (TaskRunner pid=2281424) 'reward_manager': 'naive', (TaskRunner pid=2281424) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2281424) 'cudagraph_capture_sizes': None, (TaskRunner pid=2281424) 'data_parallel_size': 1, (TaskRunner pid=2281424) 'disable_log_stats': True, (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'enable_chunked_prefill': True, (TaskRunner pid=2281424) 'enable_prefix_caching': True, (TaskRunner pid=2281424) 'enforce_eager': True, (TaskRunner pid=2281424) 'engine_kwargs': {}, (TaskRunner pid=2281424) 'expert_parallel_size': 1, (TaskRunner pid=2281424) 'free_cache_engine': True, (TaskRunner pid=2281424) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2281424) 'limit_images': None, (TaskRunner pid=2281424) 'load_format': 'auto', (TaskRunner pid=2281424) 'max_model_len': None, (TaskRunner pid=2281424) 'max_num_batched_tokens': 8192, (TaskRunner pid=2281424) 'max_num_seqs': 1024, (TaskRunner pid=2281424) 'name': '???', (TaskRunner pid=2281424) 'prompt_length': 2048, (TaskRunner pid=2281424) 'response_length': 2048, (TaskRunner pid=2281424) 'skip_tokenizer_init': False, (TaskRunner pid=2281424) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2281424) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2281424) 'memory_limit_mb': 1024, (TaskRunner pid=2281424) 'url': None}, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_dynamic_bsz': False, (TaskRunner pid=2281424) 'use_reward_loop': False}, (TaskRunner pid=2281424) 'trainer': {'balance_batch': True, (TaskRunner pid=2281424) 'critic_warmup': 0, (TaskRunner pid=2281424) 'default_hdfs_dir': None, (TaskRunner pid=2281424) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2281424) 'del_local_ckpt_after_load': False, (TaskRunner pid=2281424) 'device': 'cuda', (TaskRunner pid=2281424) 'esi_redundant_time': 0, (TaskRunner pid=2281424) 'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2281424) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2281424) 'log_val_generations': 0, (TaskRunner pid=2281424) 'logger': ['console', 'wandb'], (TaskRunner pid=2281424) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2281424) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2281424) 'n_gpus_per_node': 8, (TaskRunner pid=2281424) 'nnodes': 1, (TaskRunner pid=2281424) 'project_name': 'SDPO-root', (TaskRunner pid=2281424) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2281424) 'resume_from_path': None, (TaskRunner pid=2281424) 'resume_mode': 'auto', (TaskRunner pid=2281424) 'rollout_data_dir': None, (TaskRunner pid=2281424) 'save_freq': 10, (TaskRunner pid=2281424) 'test_freq': 10, (TaskRunner pid=2281424) 'total_epochs': 30, (TaskRunner pid=2281424) 'total_training_steps': 100, (TaskRunner pid=2281424) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2281424) 'val_before_train': False, (TaskRunner pid=2281424) 'val_only': False, (TaskRunner pid=2281424) 'validation_data_dir': None}, (TaskRunner pid=2281424) 'transfer_queue': {'enable': False}, (TaskRunner pid=2281424) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2281424) 'dir': '/users/root/SDPO', (TaskRunner pid=2281424) 'log_dir': '/users/root/output', (TaskRunner pid=2281424) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2281424) use_critic=need_critic(config), (TaskRunner pid=2281424) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2281424) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2281424) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2281424) Using dataset class: RLHFDataset (TaskRunner pid=2281424) dataset len: 720 (TaskRunner pid=2281424) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2281424) WARNING:2026-07-03 03:19:03,686:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2281424) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2281424) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2281424) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 122.84 examples/s] (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2281424) self.use_critic = need_critic(self.config) (WorkerDict pid=2281814) [W703 03:19:12.382603555 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53237 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2281810) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2281810) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2281817) [W703 03:19:13.439209755 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53237 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2281810) Model config after override: Qwen3Config { (WorkerDict pid=2281810) "architectures": [ (WorkerDict pid=2281810) "Qwen3ForCausalLM" (WorkerDict pid=2281810) ], (WorkerDict pid=2281810) "attention_bias": false, (WorkerDict pid=2281810) "attention_dropout": 0.0, (WorkerDict pid=2281810) "dtype": "bfloat16", (WorkerDict pid=2281810) "eos_token_id": 151645, (WorkerDict pid=2281810) "head_dim": 128, (WorkerDict pid=2281810) "hidden_act": "silu", (WorkerDict pid=2281810) "hidden_size": 4096, (WorkerDict pid=2281810) "initializer_range": 0.02, (WorkerDict pid=2281810) "intermediate_size": 12288, (WorkerDict pid=2281810) "layer_types": [ (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention" (WorkerDict pid=2281810) ], (WorkerDict pid=2281810) "max_position_embeddings": 40960, (WorkerDict pid=2281810) "max_window_layers": 36, (WorkerDict pid=2281810) "model_type": "qwen3", (WorkerDict pid=2281810) "num_attention_heads": 32, (WorkerDict pid=2281810) "num_hidden_layers": 36, (WorkerDict pid=2281810) "num_key_value_heads": 8, (WorkerDict pid=2281810) "pad_token_id": 151643, (WorkerDict pid=2281810) "rms_norm_eps": 1e-06, (WorkerDict pid=2281810) "rope_scaling": null, (WorkerDict pid=2281810) "rope_theta": 1000000, (WorkerDict pid=2281810) "sliding_window": null, (WorkerDict pid=2281810) "tie_word_embeddings": false, (WorkerDict pid=2281810) "transformers_version": "4.57.1", (WorkerDict pid=2281810) "use_cache": true, (WorkerDict pid=2281810) "use_sliding_window": false, (WorkerDict pid=2281810) "vocab_size": 151936 (WorkerDict pid=2281810) } (WorkerDict pid=2281810) (WorkerDict pid=2281810) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2281810) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2281812) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2281810) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2281811) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2281811) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2281817) Loading checkpoint shards: 80%|████████ | 4/5 [00:11<00:02, 2.89s/it] [repeated 15x across cluster] (WorkerDict pid=2281817) Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.20s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.58s/it] [repeated 7x across cluster] (WorkerDict pid=2281810) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2281810) Actor use_remove_padding=True (WorkerDict pid=2281810) Actor use_fused_kernels=False (WorkerDict pid=2281810) Actor use_prefix_grouper=False (WorkerDict pid=2281817) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2281817) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2281814) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2281814) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2281811) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2281810) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2281816) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2281816) warnings.warn( (WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2281810) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2281424) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2281817) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2281817) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2282697) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2282697) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2282700) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2282700) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2282697) ['serve', (vLLMHttpServer pid=2282697) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2282697) '--dtype', (vLLMHttpServer pid=2282697) 'bfloat16', (vLLMHttpServer pid=2282697) '--load_format', (vLLMHttpServer pid=2282697) 'dummy', (vLLMHttpServer pid=2282697) '--trust_remote_code', (vLLMHttpServer pid=2282697) '--max_model_len', (vLLMHttpServer pid=2282697) '40960', (vLLMHttpServer pid=2282697) '--max_num_seqs', (vLLMHttpServer pid=2282697) '1024', (vLLMHttpServer pid=2282697) '--enable_chunked_prefill', (vLLMHttpServer pid=2282697) '--max_num_batched_tokens', (vLLMHttpServer pid=2282697) '10240', (vLLMHttpServer pid=2282697) '--enable_prefix_caching', (vLLMHttpServer pid=2282697) '--enable_sleep_mode', (vLLMHttpServer pid=2282697) '--logprobs_mode', (vLLMHttpServer pid=2282697) 'processed_logprobs', (vLLMHttpServer pid=2282697) '--disable_custom_all_reduce', (vLLMHttpServer pid=2282697) '--gpu_memory_utilization', (vLLMHttpServer pid=2282697) '0.8', (vLLMHttpServer pid=2282697) '--disable_log_stats', (vLLMHttpServer pid=2282697) '--tensor_parallel_size', (vLLMHttpServer pid=2282697) '2', (vLLMHttpServer pid=2282697) '--seed', (vLLMHttpServer pid=2282697) '0', (vLLMHttpServer pid=2282697) '--override_generation_config', (vLLMHttpServer pid=2282697) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2282697) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2282697) '--hf_overrides', (vLLMHttpServer pid=2282697) '{}', (vLLMHttpServer pid=2282697) '--scheduling_policy', (vLLMHttpServer pid=2282697) 'fcfs'] (WorkerDict pid=2281817) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2282697) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2282697) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2282697) WARNING 07-03 03:20:04 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2281811) WARNING 07-03 03:20:12 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2282699) WARNING 07-03 03:20:05 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2281812) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2281810) 2026-07-03 03:20:26,670 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (vLLMHttpServer pid=2282699) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2282699) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2282699) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2282699) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2281814) 2026-07-03 03:20:26,669 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2281810) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $, the charge. Rearranging the formula: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substitute the known values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \times (0.50 \, \mathrm{m})^2}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2281424) step:10 - global_seqlen/min:18370 - global_seqlen/max:31673 - global_seqlen/minmax_diff:13303 - global_seqlen/balanced_min:26421 - global_seqlen/balanced_max:26444 - global_seqlen/mean:26434.75 - actor/entropy:0.32802483439445496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45646193623542786 - training/rollout_probs_diff_mean:0.0033578479196876287 - training/rollout_probs_diff_std:0.00903936568647623 - training/rollout_actor_probs_pearson_corr:0.9993472099304199 - rollout_corr/rollout_is_mean:0.9999197721481323 - rollout_corr/rollout_is_ratio_fraction_high:7.013114554865751e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.4026229109731503e-05 - rollout_corr/rollout_is_max:3.0499253273010254 - rollout_corr/rollout_is_min:0.36710941791534424 - rollout_corr/rollout_is_std:0.029133660718798637 - rollout_corr/rollout_is_eff_sample_size:0.999151830651037 - rollout_corr/rollout_is_seq_mean:0.9999980330467224 - rollout_corr/rollout_is_seq_std:0.0017985748127102852 - rollout_corr/rollout_is_seq_max:1.008771300315857 - rollout_corr/rollout_is_seq_min:0.9932235479354858 - rollout_corr/rollout_is_seq_max_deviation:0.008771300315856934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.377936966251582) - rollout_corr/training_log_ppl:np.float64(0.3002843804570148) - rollout_corr/kl:np.float64(0.0005245888745832872) - rollout_corr/k3_kl:np.float64(0.000498560940002335) - rollout_corr/rollout_ppl:np.float64(1.377146857790649) - rollout_corr/rollout_log_ppl:np.float64(0.29975979137816466) - rollout_corr/log_ppl_diff:np.float64(0.0005245890788501129) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013371909590205178) - rollout_corr/log_ppl_diff_max:np.float64(0.0069696009159088135) - rollout_corr/log_ppl_diff_min:np.float64(-0.004811465740203857) - rollout_corr/ppl_ratio:np.float64(1.0005262338090688) - rollout_corr/chi2_token:np.float64(0.0009716309141367674) - rollout_corr/chi2_seq:np.float64(1.0234597465023398) - actor/pg_loss:np.float64(2.4606240913271904e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024767978288764425) - actor/ppo_kl:np.float64(4.621054888964693e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18785973638296127) - perf/mfu/actor:np.float64(0.10560325767310388) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.76485443115234) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.58359375) - val-aux/sciknoweval/reward/std@16:np.float64(0.21906251159087287) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.67415) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19020325207669364) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.4956125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17791978642388112) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5849125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2188371622304282) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7478) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14805144079456356) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.425825) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12986404819208913) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6028874999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17818578143360303) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8051999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10766830979919102) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3753875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08819187467072624) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6146375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13509611267304375) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8500624999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07012187539545785) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3417125) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05316338904072637) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6237) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09735119649509659) - val-aux/sciknoweval/score/mean@16:np.float64(0.58359375) - val-aux/sciknoweval/score/std@16:np.float64(0.21906251159087287) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.67415) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19020325207669364) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.4956125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17791978642388112) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5849125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2188371622304282) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7478) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14805144079456356) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.425825) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12986404819208913) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6028874999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17818578143360303) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8051999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10766830979919102) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3753875) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08819187467072624) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6146375) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13509611267304375) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8500624999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07012187539545785) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3417125) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05316338904072637) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6237) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09735119649509659) - val-core/sciknoweval/acc/mean@16:np.float64(0.58359375) - val-aux/sciknoweval/acc/std@16:np.float64(0.21906251159087287) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.67415) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19020325207669364) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.4956125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17791978642388112) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5849125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2188371622304282) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7478) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14805144079456356) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.425825) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12986404819208913) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6028874999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17818578143360303) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8051999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10766830979919102) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3753875) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08819187467072624) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6146375) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13509611267304375) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8500624999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07012187539545785) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3417125) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05316338904072637) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6237) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09735119649509659) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0432113421230519) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99315) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.016507906298959766) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9690875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.053154032584486456) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9808874999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.042999991812243614) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9968875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.005405422624550276) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9512750000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06454074729263456) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9880125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.028221887660256677) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9994) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0026720778431774776) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9238625) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07328509472283329) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.992225) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014819305487201267) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.887475) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06968272194151441) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9943125) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.007892775584975463) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01252454612404108 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01252454612404108 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:556.9921875 - response_length/max:1781.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:556.9921875 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014546141028404236 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1660982128232718) - timing_s/agent_loop/generate_sequences/max:np.float64(13.751114128157496) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.805240769754164) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.751114128157496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:20.531365724280477 - timing_s/reward:0.06193559989333153 - timing_s/old_log_prob:3.251886686310172 - timing_s/adv:0.15388510562479496 - timing_s/update_actor:13.160292625427246 - timing_s/step:37.241492161527276 - timing_s/testing:116.28815667517483 - timing_s/save_checkpoint:12.811152569949627 - timing_s/stop_profile:0.00015922822058200836 - timing_per_token_ms/gen:0.1439888191617959 - timing_per_token_ms/update_actor:0.06223007889911596 - timing_per_token_ms/adv:0.0007276648427959171 - perf/total_num_tokens:211478 - perf/time_per_step:37.241492161527276 - perf/throughput:709.8198397997786 (TaskRunner pid=2281424) Training Progress: 10%|█ | 10/100 [09:39<2:02:00, 81.34s/it] (TaskRunner pid=2281424) step:11 - global_seqlen/min:20995 - global_seqlen/max:35423 - global_seqlen/minmax_diff:14428 - global_seqlen/balanced_min:27571 - global_seqlen/balanced_max:27590 - global_seqlen/mean:27580.5 - actor/entropy:0.24689078330993652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190903186798096 - training/rollout_probs_diff_mean:0.0029323738999664783 - training/rollout_probs_diff_std:0.008981931023299694 - training/rollout_actor_probs_pearson_corr:0.999229907989502 - rollout_corr/rollout_is_mean:0.9999557137489319 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.325087163830176e-05 - rollout_corr/rollout_is_max:1.9537171125411987 - rollout_corr/rollout_is_min:0.08846962451934814 - rollout_corr/rollout_is_std:0.0269706342369318 - rollout_corr/rollout_is_eff_sample_size:0.9992731136632474 - rollout_corr/rollout_is_seq_mean:1.0000543594360352 - rollout_corr/rollout_is_seq_std:0.0017303781351074576 - rollout_corr/rollout_is_seq_max:1.0074354410171509 - rollout_corr/rollout_is_seq_min:0.9947044253349304 - rollout_corr/rollout_is_seq_max_deviation:0.007435441017150879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710583903826773) - rollout_corr/training_log_ppl:np.float64(0.22995059203822166) - rollout_corr/kl:np.float64(0.0004979196115613149) - rollout_corr/k3_kl:np.float64(0.00043510435352800414) - rollout_corr/rollout_ppl:np.float64(1.270424968097359) - rollout_corr/rollout_log_ppl:np.float64(0.22945267148315907) - rollout_corr/log_ppl_diff:np.float64(0.000497920555062592) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013768301578238606) - rollout_corr/log_ppl_diff_max:np.float64(0.006137266755104065) - rollout_corr/log_ppl_diff_min:np.float64(-0.007355779409408569) - rollout_corr/ppl_ratio:np.float64(1.000499751418829) - rollout_corr/chi2_token:np.float64(0.0007434843573719263) - rollout_corr/chi2_seq:np.float64(0.47075980668887496) - actor/pg_loss:np.float64(4.087574779987335e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001813313299408037) - actor/ppo_kl:np.float64(0.00013543530537063475) - actor/pg_clipfrac_lower:np.float64(4.278477717889473e-06) - actor/grad_norm:np.float64(0.17897598445415497) - perf/mfu/actor:np.float64(0.10733546751479987) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.49744415283203) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004445641301572323 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.004445641301572323 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:587.390625 - response_length/max:1781.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.390625 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.45193862915039e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9449637047946453) - timing_s/agent_loop/generate_sequences/max:np.float64(13.539711721241474) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.947522593545727) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.539711721241474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1671 - timing_s/gen:20.11647711135447 - timing_s/reward:0.06534852646291256 - timing_s/old_log_prob:3.4682928640395403 - timing_s/adv:0.13045809976756573 - timing_s/update_actor:13.693839404731989 - timing_s/step:37.579134887084365 - timing_s/stop_profile:0.00012080185115337372 - timing_per_token_ms/gen:0.13377807777614495 - timing_per_token_ms/update_actor:0.06206304909597356 - timing_per_token_ms/adv:0.0005912605816046016 - perf/total_num_tokens:220644 - perf/time_per_step:37.579134887084365 - perf/throughput:733.9312116383815 (TaskRunner pid=2281424) Training Progress: 11%|█ | 11/100 [10:17<1:40:48, 67.96s/it] (TaskRunner pid=2281424) step:12 - global_seqlen/min:18284 - global_seqlen/max:36197 - global_seqlen/minmax_diff:17913 - global_seqlen/balanced_min:27117 - global_seqlen/balanced_max:27742 - global_seqlen/mean:27219.25 - actor/entropy:0.2918388545513153 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463563919067383 - training/rollout_probs_diff_mean:0.00314357434399426 - training/rollout_probs_diff_std:0.008670532144606113 - training/rollout_actor_probs_pearson_corr:0.9993590712547302 - rollout_corr/rollout_is_mean:0.9999263882637024 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.328426969848806e-05 - rollout_corr/rollout_is_max:1.6952860355377197 - rollout_corr/rollout_is_min:0.2599896490573883 - rollout_corr/rollout_is_std:0.02788124419748783 - rollout_corr/rollout_is_eff_sample_size:0.9992230019996114 - rollout_corr/rollout_is_seq_mean:0.9999516010284424 - rollout_corr/rollout_is_seq_std:0.0013419243041425943 - rollout_corr/rollout_is_seq_max:1.0067940950393677 - rollout_corr/rollout_is_seq_min:0.9959049224853516 - rollout_corr/rollout_is_seq_max_deviation:0.006794095039367676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284353747498244) - rollout_corr/training_log_ppl:np.float64(0.24118711793562397) - rollout_corr/kl:np.float64(0.0004640274163274505) - rollout_corr/k3_kl:np.float64(0.000438987553025072) - rollout_corr/rollout_ppl:np.float64(1.2837417596019804) - rollout_corr/rollout_log_ppl:np.float64(0.2407230889366474) - rollout_corr/log_ppl_diff:np.float64(0.00046402899897657335) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012675680045504123) - rollout_corr/log_ppl_diff_max:np.float64(0.00536249577999115) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736485123634338) - rollout_corr/ppl_ratio:np.float64(1.0004653770010918) - rollout_corr/chi2_token:np.float64(0.0008103374857455492) - rollout_corr/chi2_seq:np.float64(0.7635552103165537) - actor/pg_loss:np.float64(6.2944600358605385e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022421280800699606) - actor/ppo_kl:np.float64(8.907836768656807e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1681164726614952) - perf/mfu/actor:np.float64(0.10657234495140916) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.30431365966797) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02310134656727314 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02310134656727314 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:588.1015625 - response_length/max:3570.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:588.1015625 - response_length_non_aborted/max:3570.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.758848369121552e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9172126762568951) - timing_s/agent_loop/generate_sequences/max:np.float64(22.92699559032917) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.20948198466067) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.92699559032917) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:3570 - timing_s/gen:29.500037664547563 - timing_s/reward:0.06999753788113594 - timing_s/old_log_prob:3.321774547919631 - timing_s/adv:0.16080131568014622 - timing_s/update_actor:13.495587630197406 - timing_s/step:46.61714624799788 - timing_s/stop_profile:0.00011683814227581024 - timing_per_token_ms/gen:0.19594323408576034 - timing_per_token_ms/update_actor:0.06197630183692334 - timing_per_token_ms/adv:0.0007384540154492969 - perf/total_num_tokens:217754 - perf/time_per_step:46.61714624799788 - perf/throughput:583.8892379897454 (TaskRunner pid=2281424) Training Progress: 12%|█▏ | 12/100 [11:03<1:30:10, 61.48s/it] (TaskRunner pid=2281424) step:13 - global_seqlen/min:24964 - global_seqlen/max:33451 - global_seqlen/minmax_diff:8487 - global_seqlen/balanced_min:28949 - global_seqlen/balanced_max:28972 - global_seqlen/mean:28959.375 - actor/entropy:0.31463027000427246 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24114426970481873 - training/rollout_probs_diff_mean:0.00320067978464067 - training/rollout_probs_diff_std:0.008584943599998951 - training/rollout_actor_probs_pearson_corr:0.9994125962257385 - rollout_corr/rollout_is_mean:1.0001263618469238 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.238653440959752e-06 - rollout_corr/rollout_is_max:1.859130859375 - rollout_corr/rollout_is_min:0.4844902455806732 - rollout_corr/rollout_is_std:0.02811329998075962 - rollout_corr/rollout_is_eff_sample_size:0.9992105046242893 - rollout_corr/rollout_is_seq_mean:1.0000969171524048 - rollout_corr/rollout_is_seq_std:0.0011731568956747651 - rollout_corr/rollout_is_seq_max:1.0035706758499146 - rollout_corr/rollout_is_seq_min:0.9957525134086609 - rollout_corr/rollout_is_seq_max_deviation:0.004247486591339111 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3780980156734586) - rollout_corr/training_log_ppl:np.float64(0.2991240085102618) - rollout_corr/kl:np.float64(0.0003781683996528784) - rollout_corr/k3_kl:np.float64(0.0004593968702977236) - rollout_corr/rollout_ppl:np.float64(1.3775346302427351) - rollout_corr/rollout_log_ppl:np.float64(0.2987458378338488) - rollout_corr/log_ppl_diff:np.float64(0.0003781706764129922) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010912520374404266) - rollout_corr/log_ppl_diff_max:np.float64(0.004893004894256592) - rollout_corr/log_ppl_diff_min:np.float64(-0.003899678587913513) - rollout_corr/ppl_ratio:np.float64(1.0003791959024966) - rollout_corr/chi2_token:np.float64(0.001073798630386591) - rollout_corr/chi2_seq:np.float64(0.506177173461765) - actor/pg_loss:np.float64(3.237277269363403e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004119513098430616) - actor/ppo_kl:np.float64(7.283486395426308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21577078476548195) - perf/mfu/actor:np.float64(0.11467366616050297) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.31635284423828) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002386284904787317 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002386284904787317 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:626.13671875 - response_length/max:1861.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:626.13671875 - response_length_non_aborted/max:1861.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013312511146068573 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9897179938852787) - timing_s/agent_loop/generate_sequences/max:np.float64(15.492640441283584) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.851898552646162) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.492640441283584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1861 - timing_s/gen:22.31190705113113 - timing_s/reward:0.0678411666303873 - timing_s/old_log_prob:3.3336705192923546 - timing_s/adv:0.18029628694057465 - timing_s/update_actor:13.361042624339461 - timing_s/step:39.349795043468475 - timing_s/stop_profile:0.00011570192873477936 - timing_per_token_ms/gen:0.1391962558791893 - timing_per_token_ms/update_actor:0.05767149077086203 - timing_per_token_ms/adv:0.0007782293598384575 - perf/total_num_tokens:231675 - perf/time_per_step:39.349795043468475 - perf/throughput:735.9472893825621 (TaskRunner pid=2281424) Training Progress: 13%|█▎ | 13/100 [11:43<1:19:27, 54.80s/it] (TaskRunner pid=2281424) step:14 - global_seqlen/min:18482 - global_seqlen/max:39950 - global_seqlen/minmax_diff:21468 - global_seqlen/balanced_min:27989 - global_seqlen/balanced_max:28043 - global_seqlen/mean:28025.875 - actor/entropy:0.27694693207740784 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9996616840362549 - training/rollout_probs_diff_mean:0.0032118367962539196 - training/rollout_probs_diff_std:0.009403628297150135 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999675154685974 - rollout_corr/rollout_is_ratio_fraction_high:2.5921677661244757e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.2402098440798e-05 - rollout_corr/rollout_is_max:2.0761139392852783 - rollout_corr/rollout_is_min:0.000335350981913507 - rollout_corr/rollout_is_std:0.028911858797073364 - rollout_corr/rollout_is_eff_sample_size:0.9991645645790151 - rollout_corr/rollout_is_seq_mean:0.9999456405639648 - rollout_corr/rollout_is_seq_std:0.001373259350657463 - rollout_corr/rollout_is_seq_max:1.0052764415740967 - rollout_corr/rollout_is_seq_min:0.995773434638977 - rollout_corr/rollout_is_seq_max_deviation:0.00527644157409668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3075349098071456) - rollout_corr/training_log_ppl:np.float64(0.2546500996249961) - rollout_corr/kl:np.float64(0.0006805974277988014) - rollout_corr/k3_kl:np.float64(0.0006370248192979489) - rollout_corr/rollout_ppl:np.float64(1.3065974819473922) - rollout_corr/rollout_log_ppl:np.float64(0.2539695030573057) - rollout_corr/log_ppl_diff:np.float64(0.0006805965676903725) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013074282323941588) - rollout_corr/log_ppl_diff_max:np.float64(0.01114201545715332) - rollout_corr/log_ppl_diff_min:np.float64(-0.004195064306259155) - rollout_corr/ppl_ratio:np.float64(1.000682536046952) - rollout_corr/chi2_token:np.float64(0.0010629112366586924) - rollout_corr/chi2_seq:np.float64(0.7319330512546003) - actor/pg_loss:np.float64(-7.280893623828888e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004904413003714581) - actor/ppo_kl:np.float64(0.00013513596405800854) - actor/pg_clipfrac_lower:np.float64(5.243288796918932e-06) - actor/grad_norm:np.float64(0.22375838086009026) - perf/mfu/actor:np.float64(0.11024404131767285) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.50100326538086) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004044591914862394 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004044591914862394 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:602.77734375 - response_length/max:2091.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:602.77734375 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.801425039768219e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1079022623598576) - timing_s/agent_loop/generate_sequences/max:np.float64(15.903715757653117) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.53672656758863) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.903715757653117) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:249 - timing_s/agent_loop/slowest/response_length:2091 - timing_s/gen:22.835159683600068 - timing_s/reward:0.07095035165548325 - timing_s/old_log_prob:3.195645259693265 - timing_s/adv:0.15754373371601105 - timing_s/update_actor:13.296314042061567 - timing_s/step:39.66534601524472 - timing_s/stop_profile:0.00011271052062511444 - timing_per_token_ms/gen:0.14798141210672 - timing_per_token_ms/update_actor:0.0593037418192187 - timing_per_token_ms/adv:0.0007026708966089866 - perf/total_num_tokens:224207 - perf/time_per_step:39.66534601524472 - perf/throughput:706.5581878254312 (TaskRunner pid=2281424) Training Progress: 14%|█▍ | 14/100 [12:23<1:12:00, 50.24s/it] (TaskRunner pid=2281424) step:15 - global_seqlen/min:18409 - global_seqlen/max:34972 - global_seqlen/minmax_diff:16563 - global_seqlen/balanced_min:27584 - global_seqlen/balanced_max:27657 - global_seqlen/mean:27632.0 - actor/entropy:0.23792505264282227 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2081579566001892 - training/rollout_probs_diff_mean:0.0028373654931783676 - training/rollout_probs_diff_std:0.008734497241675854 - training/rollout_actor_probs_pearson_corr:0.9992759227752686 - rollout_corr/rollout_is_mean:1.000104308128357 - rollout_corr/rollout_is_ratio_fraction_high:6.5414201344538014e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0567126274108887 - rollout_corr/rollout_is_min:0.6020901799201965 - rollout_corr/rollout_is_std:0.0269706342369318 - rollout_corr/rollout_is_eff_sample_size:0.9992733517354029 - rollout_corr/rollout_is_seq_mean:1.0001739263534546 - rollout_corr/rollout_is_seq_std:0.0016109951538965106 - rollout_corr/rollout_is_seq_max:1.0077435970306396 - rollout_corr/rollout_is_seq_min:0.9901288747787476 - rollout_corr/rollout_is_seq_max_deviation:0.009871125221252441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2825340307317674) - rollout_corr/training_log_ppl:np.float64(0.2307681181409862) - rollout_corr/kl:np.float64(0.00024018173945883348) - rollout_corr/k3_kl:np.float64(0.00046571865181022076) - rollout_corr/rollout_ppl:np.float64(1.2822559108026326) - rollout_corr/rollout_log_ppl:np.float64(0.23052793518581893) - rollout_corr/log_ppl_diff:np.float64(0.0002401829551672563) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012830473860958591) - rollout_corr/log_ppl_diff_max:np.float64(0.011005207896232605) - rollout_corr/log_ppl_diff_min:np.float64(-0.004439152777194977) - rollout_corr/ppl_ratio:np.float64(1.0002417913638055) - rollout_corr/chi2_token:np.float64(0.0013833614066243172) - rollout_corr/chi2_seq:np.float64(0.8579549191053957) - actor/pg_loss:np.float64(0.00013076758477836847) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022705169044456852) - actor/ppo_kl:np.float64(1.2435915886399584e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1974821835756302) - perf/mfu/actor:np.float64(0.10945352184092855) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.5802116394043) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011041100136935711 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011041100136935711 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:597.15625 - response_length/max:2544.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:597.15625 - response_length_non_aborted/max:2544.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001034904271364212 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0850455239415169) - timing_s/agent_loop/generate_sequences/max:np.float64(18.34066752716899) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.139947620045859) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.34066752716899) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:2544 - timing_s/gen:24.80101560242474 - timing_s/reward:0.06913194060325623 - timing_s/old_log_prob:3.2432605400681496 - timing_s/adv:0.15737476013600826 - timing_s/update_actor:13.316460059955716 - timing_s/step:41.67461157031357 - timing_s/stop_profile:0.00011474825441837311 - timing_per_token_ms/gen:0.1622338662568995 - timing_per_token_ms/update_actor:0.06024021089658601 - timing_per_token_ms/adv:0.0007119225903662794 - perf/total_num_tokens:221056 - perf/time_per_step:41.67461157031357 - perf/throughput:663.0415727661715 (TaskRunner pid=2281424) Training Progress: 15%|█▌ | 15/100 [13:04<1:07:32, 47.68s/it] (TaskRunner pid=2281424) step:16 - global_seqlen/min:19774 - global_seqlen/max:37148 - global_seqlen/minmax_diff:17374 - global_seqlen/balanced_min:26832 - global_seqlen/balanced_max:26869 - global_seqlen/mean:26857.125 - actor/entropy:0.269243985414505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29323524236679077 - training/rollout_probs_diff_mean:0.0030834192875772715 - training/rollout_probs_diff_std:0.009029646404087543 - training/rollout_actor_probs_pearson_corr:0.9992733001708984 - rollout_corr/rollout_is_mean:1.000123381614685 - rollout_corr/rollout_is_ratio_fraction_high:1.3476997992256656e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.39180064201355 - rollout_corr/rollout_is_min:0.5188149213790894 - rollout_corr/rollout_is_std:0.028928346931934357 - rollout_corr/rollout_is_eff_sample_size:0.9991640885384842 - rollout_corr/rollout_is_seq_mean:1.0000724792480469 - rollout_corr/rollout_is_seq_std:0.0016198219964280725 - rollout_corr/rollout_is_seq_max:1.0117334127426147 - rollout_corr/rollout_is_seq_min:0.9945037961006165 - rollout_corr/rollout_is_seq_max_deviation:0.011733412742614746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2698871567845345) - rollout_corr/training_log_ppl:np.float64(0.22661357806646265) - rollout_corr/kl:np.float64(0.000278438638499523) - rollout_corr/k3_kl:np.float64(0.0004574081480726022) - rollout_corr/rollout_ppl:np.float64(1.2695180620066822) - rollout_corr/rollout_log_ppl:np.float64(0.22633513917389791) - rollout_corr/log_ppl_diff:np.float64(0.0002784388925647363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010494471789570525) - rollout_corr/log_ppl_diff_max:np.float64(0.00465761125087738) - rollout_corr/log_ppl_diff_min:np.float64(-0.009550660848617554) - rollout_corr/ppl_ratio:np.float64(1.000279544852674) - rollout_corr/chi2_token:np.float64(0.0013024122454226017) - rollout_corr/chi2_seq:np.float64(0.9144449539016932) - actor/pg_loss:np.float64(-1.0943855158984661e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023535769855698163) - actor/ppo_kl:np.float64(-7.823348023361376e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21825583279132843) - perf/mfu/actor:np.float64(0.10663394115254357) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.46512222290039) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005873444024473429 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005873444024473429 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:579.69140625 - response_length/max:1869.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:579.69140625 - response_length_non_aborted/max:1869.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001010652631521225 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3836285527795553) - timing_s/agent_loop/generate_sequences/max:np.float64(14.655153315514326) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.240751736462698) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.655153315514326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1869 - timing_s/gen:21.27606852352619 - timing_s/reward:0.06742825172841549 - timing_s/old_log_prob:3.215101270005107 - timing_s/adv:0.13886815682053566 - timing_s/update_actor:13.220384620130062 - timing_s/step:38.01398337073624 - timing_s/stop_profile:2.8956681489944458e-05 - timing_per_token_ms/gen:0.14336876788920688 - timing_per_token_ms/update_actor:0.06153108635106169 - timing_per_token_ms/adv:0.0006463282872819394 - perf/total_num_tokens:214857 - perf/time_per_step:38.01398337073624 - perf/throughput:706.5064646888607 (TaskRunner pid=2281424) Training Progress: 16%|█▌ | 16/100 [13:42<1:02:41, 44.77s/it] (TaskRunner pid=2281424) step:17 - global_seqlen/min:21078 - global_seqlen/max:38139 - global_seqlen/minmax_diff:17061 - global_seqlen/balanced_min:27815 - global_seqlen/balanced_max:28195 - global_seqlen/mean:27876.125 - actor/entropy:0.3084404766559601 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105865716934204 - training/rollout_probs_diff_mean:0.003280392149463296 - training/rollout_probs_diff_std:0.008956593461334705 - training/rollout_actor_probs_pearson_corr:0.9993877410888672 - rollout_corr/rollout_is_mean:1.0001431703567505 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.458905318140751e-06 - rollout_corr/rollout_is_max:1.8297889232635498 - rollout_corr/rollout_is_min:0.49442058801651 - rollout_corr/rollout_is_std:0.02931314706802368 - rollout_corr/rollout_is_eff_sample_size:0.9991418341499507 - rollout_corr/rollout_is_seq_mean:1.0001916885375977 - rollout_corr/rollout_is_seq_std:0.0019043951760977507 - rollout_corr/rollout_is_seq_max:1.009162425994873 - rollout_corr/rollout_is_seq_min:0.9893993139266968 - rollout_corr/rollout_is_seq_max_deviation:0.010600686073303223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3411333807744086) - rollout_corr/training_log_ppl:np.float64(0.2804811333771795) - rollout_corr/kl:np.float64(0.00039379035426634346) - rollout_corr/k3_kl:np.float64(0.0005168410488067821) - rollout_corr/rollout_ppl:np.float64(1.3405519826337695) - rollout_corr/rollout_log_ppl:np.float64(0.28008734311151784) - rollout_corr/log_ppl_diff:np.float64(0.0003937902656616643) - rollout_corr/log_ppl_abs_diff:np.float64(0.001501921404269524) - rollout_corr/log_ppl_diff_max:np.float64(0.011711418628692627) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628694176673889) - rollout_corr/ppl_ratio:np.float64(1.0003962186165154) - rollout_corr/chi2_token:np.float64(0.001294472487643361) - rollout_corr/chi2_seq:np.float64(1.0754117609467357) - actor/pg_loss:np.float64(-0.00010975566692650318) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020725219656014815) - actor/ppo_kl:np.float64(0.00011342834537941826) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2514360435307026) - perf/mfu/actor:np.float64(0.10841431290124035) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.30791854858398) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012801550328731537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012801550328731537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.78515625 - response_length/max:2740.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.78515625 - response_length_non_aborted/max:2740.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.358139216899872e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0903181321918964) - timing_s/agent_loop/generate_sequences/max:np.float64(20.050666142255068) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.366174238610256) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.050666142255068) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:2740 - timing_s/gen:27.158069726079702 - timing_s/reward:0.08344477787613869 - timing_s/old_log_prob:3.2945270854979753 - timing_s/adv:0.13789121992886066 - timing_s/update_actor:13.660845940932631 - timing_s/step:44.418967282399535 - timing_s/stop_profile:2.7371570467948914e-05 - timing_per_token_ms/gen:0.17541139819848023 - timing_per_token_ms/update_actor:0.06125692658562045 - timing_per_token_ms/adv:0.0006183213230356652 - perf/total_num_tokens:223009 - perf/time_per_step:44.418967282399535 - perf/throughput:627.5725597755077 (TaskRunner pid=2281424) Training Progress: 17%|█▋ | 17/100 [14:27<1:01:48, 44.68s/it] (TaskRunner pid=2281424) step:18 - global_seqlen/min:18777 - global_seqlen/max:44614 - global_seqlen/minmax_diff:25837 - global_seqlen/balanced_min:32860 - global_seqlen/balanced_max:32946 - global_seqlen/mean:32927.0 - actor/entropy:0.2818203866481781 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2900807857513428 - training/rollout_probs_diff_mean:0.002938726684078574 - training/rollout_probs_diff_std:0.00846165232360363 - training/rollout_actor_probs_pearson_corr:0.9994096159934998 - rollout_corr/rollout_is_mean:1.0000146627426147 - rollout_corr/rollout_is_ratio_fraction_high:5.105583568365546e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.552791738708038e-05 - rollout_corr/rollout_is_max:2.0839157104492188 - rollout_corr/rollout_is_min:0.4158320426940918 - rollout_corr/rollout_is_std:0.02699493244290352 - rollout_corr/rollout_is_eff_sample_size:0.9992719233041721 - rollout_corr/rollout_is_seq_mean:1.0000157356262207 - rollout_corr/rollout_is_seq_std:0.0014222179306671023 - rollout_corr/rollout_is_seq_max:1.0088167190551758 - rollout_corr/rollout_is_seq_min:0.995079755783081 - rollout_corr/rollout_is_seq_max_deviation:0.008816719055175781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.289564796257764) - rollout_corr/training_log_ppl:np.float64(0.2442400810541585) - rollout_corr/kl:np.float64(0.0005266999699861663) - rollout_corr/k3_kl:np.float64(0.0004649700730112727) - rollout_corr/rollout_ppl:np.float64(1.2888782764784992) - rollout_corr/rollout_log_ppl:np.float64(0.24371338007040322) - rollout_corr/log_ppl_diff:np.float64(0.0005267009837552905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011411418090574443) - rollout_corr/log_ppl_diff_max:np.float64(0.010496377944946289) - rollout_corr/log_ppl_diff_min:np.float64(-0.007471442222595215) - rollout_corr/ppl_ratio:np.float64(1.0005280999466777) - rollout_corr/chi2_token:np.float64(0.0008352911099791527) - rollout_corr/chi2_seq:np.float64(0.4274897330906242) - actor/pg_loss:np.float64(9.232200682163239e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002937083892220471) - actor/ppo_kl:np.float64(0.00015827277887936475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1933615617454052) - perf/mfu/actor:np.float64(0.12426924001280201) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.3476791381836) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007050810847431421 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007050810847431421 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:765.09375 - response_length/max:3008.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:765.09375 - response_length_non_aborted/max:3008.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.749613046646118e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9996219798922539) - timing_s/agent_loop/generate_sequences/max:np.float64(21.299407305195928) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.81618346637697) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.299407305195928) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:3008 - timing_s/gen:28.408122271299362 - timing_s/reward:0.07211213000118732 - timing_s/old_log_prob:3.439273815602064 - timing_s/adv:0.14084554836153984 - timing_s/update_actor:14.02409834600985 - timing_s/step:46.15811971202493 - timing_s/stop_profile:0.0001232251524925232 - timing_per_token_ms/gen:0.14504003937068252 - timing_per_token_ms/update_actor:0.05323935655392934 - timing_per_token_ms/adv:0.0005346886611350102 - perf/total_num_tokens:263416 - perf/time_per_step:46.15811971202493 - perf/throughput:713.352281363012 (TaskRunner pid=2281424) Training Progress: 18%|█▊ | 18/100 [15:13<1:01:41, 45.14s/it] (TaskRunner pid=2281424) step:19 - global_seqlen/min:21194 - global_seqlen/max:44487 - global_seqlen/minmax_diff:23293 - global_seqlen/balanced_min:30623 - global_seqlen/balanced_max:30828 - global_seqlen/mean:30657.0 - actor/entropy:0.2834802567958832 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35189008712768555 - training/rollout_probs_diff_mean:0.002977204043418169 - training/rollout_probs_diff_std:0.00879612471908331 - training/rollout_actor_probs_pearson_corr:0.9994228482246399 - rollout_corr/rollout_is_mean:1.0000439882278442 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2517451725434512e-05 - rollout_corr/rollout_is_max:1.6279520988464355 - rollout_corr/rollout_is_min:0.2959691882133484 - rollout_corr/rollout_is_std:0.027716144919395447 - rollout_corr/rollout_is_eff_sample_size:0.9992325240192108 - rollout_corr/rollout_is_seq_mean:1.000134825706482 - rollout_corr/rollout_is_seq_std:0.0016275998204946518 - rollout_corr/rollout_is_seq_max:1.009211540222168 - rollout_corr/rollout_is_seq_min:0.9948177337646484 - rollout_corr/rollout_is_seq_max_deviation:0.009211540222167969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2855765600688756) - rollout_corr/training_log_ppl:np.float64(0.23977286802255549) - rollout_corr/kl:np.float64(0.0005027599546281181) - rollout_corr/k3_kl:np.float64(0.0004755220272727456) - rollout_corr/rollout_ppl:np.float64(1.2849308247677982) - rollout_corr/rollout_log_ppl:np.float64(0.2392701047792798) - rollout_corr/log_ppl_diff:np.float64(0.0005027632432756945) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011823297390947118) - rollout_corr/log_ppl_diff_max:np.float64(0.009016215801239014) - rollout_corr/log_ppl_diff_min:np.float64(-0.008047908544540405) - rollout_corr/ppl_ratio:np.float64(1.0005042462144047) - rollout_corr/chi2_token:np.float64(0.0009039849974215031) - rollout_corr/chi2_seq:np.float64(0.45300188730470836) - actor/pg_loss:np.float64(8.254800923168659e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003364918597981159) - actor/ppo_kl:np.float64(0.0002041434732245051) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19276084750890732) - perf/mfu/actor:np.float64(0.11752787272188393) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.36764144897461) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0269034281373024 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0269034281373024 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:693.90625 - response_length/max:3216.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:693.90625 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013180077075958252 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2591454926878214) - timing_s/agent_loop/generate_sequences/max:np.float64(22.810867067426443) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.210481837464613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.810867067426443) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:3216 - timing_s/gen:29.960727905854583 - timing_s/reward:0.07135201431810856 - timing_s/old_log_prob:3.375285627320409 - timing_s/adv:0.1431940197944641 - timing_s/update_actor:13.87465051561594 - timing_s/step:47.520265359431505 - timing_s/stop_profile:0.00011952407658100128 - timing_per_token_ms/gen:0.16865980581994247 - timing_per_token_ms/update_actor:0.05657211450735534 - timing_per_token_ms/adv:0.0005838553176862712 - perf/total_num_tokens:245256 - perf/time_per_step:47.520265359431505 - perf/throughput:645.1352863482149 (TaskRunner pid=2281424) Training Progress: 19%|█▉ | 19/100 [16:01<1:01:55, 45.87s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The electric field due to a point charge is given by: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for the charge $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substituting the given values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2281424) step:20 - global_seqlen/min:19499 - global_seqlen/max:36025 - global_seqlen/minmax_diff:16526 - global_seqlen/balanced_min:27704 - global_seqlen/balanced_max:28173 - global_seqlen/mean:27765.0 - actor/entropy:0.280457079410553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135375022888184 - training/rollout_probs_diff_mean:0.003101746318861842 - training/rollout_probs_diff_std:0.008900674059987068 - training/rollout_actor_probs_pearson_corr:0.999406099319458 - rollout_corr/rollout_is_mean:0.9998956918716431 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9279214029666036e-05 - rollout_corr/rollout_is_max:1.8279300928115845 - rollout_corr/rollout_is_min:0.40598663687705994 - rollout_corr/rollout_is_std:0.028816867619752884 - rollout_corr/rollout_is_eff_sample_size:0.9991700390777238 - rollout_corr/rollout_is_seq_mean:0.999803900718689 - rollout_corr/rollout_is_seq_std:0.001518259639851749 - rollout_corr/rollout_is_seq_max:1.0038286447525024 - rollout_corr/rollout_is_seq_min:0.9914582967758179 - rollout_corr/rollout_is_seq_max_deviation:0.008541703224182129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2934490512125194) - rollout_corr/training_log_ppl:np.float64(0.24451396349468268) - rollout_corr/kl:np.float64(0.0004959776629758217) - rollout_corr/k3_kl:np.float64(0.0004771725971863816) - rollout_corr/rollout_ppl:np.float64(1.2928053527139127) - rollout_corr/rollout_log_ppl:np.float64(0.2440179859258933) - rollout_corr/log_ppl_diff:np.float64(0.0004959775687893853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011907425214303657) - rollout_corr/log_ppl_diff_max:np.float64(0.009751290082931519) - rollout_corr/log_ppl_diff_min:np.float64(-0.004419475793838501) - rollout_corr/ppl_ratio:np.float64(1.0004974079784006) - rollout_corr/chi2_token:np.float64(0.0009594245348125696) - rollout_corr/chi2_seq:np.float64(0.41463141702115536) - actor/pg_loss:np.float64(1.230929046869278e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021071931905680685) - actor/ppo_kl:np.float64(-0.0001457839747551759) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1746744140982628) - perf/mfu/actor:np.float64(0.10426063797607768) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.34674835205078) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59921875) - val-aux/sciknoweval/reward/std@16:np.float64(0.20532643626967628) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.684225) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18261389709388515) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.514475) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1643688800388951) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.60015) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2055792779718459) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7575125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14221272724332604) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4511375) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11570816324542939) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6215249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17291949688631747) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8132375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09937488352535821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4064375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07134373339436899) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6354124999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1400502136179348) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8518374999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06232857449090168) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.37925) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04032492559816903) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6459375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10760674780586772) - val-aux/sciknoweval/score/mean@16:np.float64(0.59921875) - val-aux/sciknoweval/score/std@16:np.float64(0.20532643626967628) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.684225) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18261389709388515) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.514475) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1643688800388951) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.60015) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2055792779718459) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7575125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14221272724332604) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4511375) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11570816324542939) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6215249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17291949688631747) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8132375) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09937488352535821) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4064375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07134373339436899) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6354124999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1400502136179348) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8518374999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06232857449090168) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.37925) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04032492559816903) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6459375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10760674780586772) - val-core/sciknoweval/acc/mean@16:np.float64(0.59921875) - val-aux/sciknoweval/acc/std@16:np.float64(0.20532643626967628) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.684225) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18261389709388515) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.514475) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1643688800388951) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.60015) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2055792779718459) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7575125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14221272724332604) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4511375) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11570816324542939) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6215249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17291949688631747) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8132375) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09937488352535821) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4064375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07134373339436899) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6354124999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1400502136179348) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8518374999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06232857449090168) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.37925) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04032492559816903) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6459375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10760674780586772) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9578125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08158753118437541) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9860749999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.03951222065525328) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9282874999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0973249508439899) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9574) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08213995093780886) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9966125) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.012924820131308547) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8906499999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.10540656140217979) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.974175) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.05962872167707507) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.999525) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0032780861692795426) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8437625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.10035091942020005) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9859375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03701549481184431) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0003950870157319778) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7979749999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08009154869844874) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9934750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.018079184472962695) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011969178915023804 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.011969178915023804 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:607.84375 - response_length/max:2668.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:607.84375 - response_length_non_aborted/max:2668.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.631311357021332e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.728585135191679) - timing_s/agent_loop/generate_sequences/max:np.float64(19.024446960538626) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.399849012530467) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.024446960538626) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:2668 - timing_s/gen:26.06310199201107 - timing_s/reward:0.06901264749467373 - timing_s/old_log_prob:3.238787254318595 - timing_s/adv:0.17001512832939625 - timing_s/update_actor:14.12741300649941 - timing_s/step:43.75453623570502 - timing_s/testing:145.7744435351342 - timing_s/save_checkpoint:13.140905741602182 - timing_s/stop_profile:0.00017121247947216034 - timing_per_token_ms/gen:0.16749204405950252 - timing_per_token_ms/update_actor:0.06360261573248428 - timing_per_token_ms/adv:0.000765420170760833 - perf/total_num_tokens:222120 - perf/time_per_step:43.75453623570502 - perf/throughput:634.5627765411652 (TaskRunner pid=2281424) Training Progress: 20%|██ | 20/100 [19:23<2:03:58, 92.98s/it] (TaskRunner pid=2281424) step:21 - global_seqlen/min:25301 - global_seqlen/max:34398 - global_seqlen/minmax_diff:9097 - global_seqlen/balanced_min:29031 - global_seqlen/balanced_max:29324 - global_seqlen/mean:29106.375 - actor/entropy:0.24065154790878296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37683379650115967 - training/rollout_probs_diff_mean:0.002658553421497345 - training/rollout_probs_diff_std:0.008330690674483776 - training/rollout_actor_probs_pearson_corr:0.9994138479232788 - rollout_corr/rollout_is_mean:1.0000982284545898 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2177079042885453e-05 - rollout_corr/rollout_is_max:1.7682775259017944 - rollout_corr/rollout_is_min:0.3936323821544647 - rollout_corr/rollout_is_std:0.02563127689063549 - rollout_corr/rollout_is_eff_sample_size:0.9993435879743411 - rollout_corr/rollout_is_seq_mean:1.0001851320266724 - rollout_corr/rollout_is_seq_std:0.0012425798922777176 - rollout_corr/rollout_is_seq_max:1.0056188106536865 - rollout_corr/rollout_is_seq_min:0.9965409636497498 - rollout_corr/rollout_is_seq_max_deviation:0.0056188106536865234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2498526307754219) - rollout_corr/training_log_ppl:np.float64(0.21081234834855422) - rollout_corr/kl:np.float64(0.00032340780622153176) - rollout_corr/k3_kl:np.float64(0.0003890872453951033) - rollout_corr/rollout_ppl:np.float64(1.2494138525798917) - rollout_corr/rollout_log_ppl:np.float64(0.21048893986153416) - rollout_corr/log_ppl_diff:np.float64(0.0003234084870200604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010021366470027715) - rollout_corr/log_ppl_diff_max:np.float64(0.004034250974655151) - rollout_corr/log_ppl_diff_min:np.float64(-0.004479348659515381) - rollout_corr/ppl_ratio:np.float64(1.0003242590464652) - rollout_corr/chi2_token:np.float64(0.0009132004342973232) - rollout_corr/chi2_seq:np.float64(0.7536862585693598) - actor/pg_loss:np.float64(3.263808321207762e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000137664294015849) - actor/ppo_kl:np.float64(0.00017732822421123728) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13653844129294157) - perf/mfu/actor:np.float64(0.11284239253639325) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.92810821533203) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021431660279631615 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021431660279631615 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:641.57421875 - response_length/max:3017.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:641.57421875 - response_length_non_aborted/max:3017.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.9451907873153687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2093537878245115) - timing_s/agent_loop/generate_sequences/max:np.float64(21.849115535616875) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.7584272636668175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.849115535616875) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:364 - timing_s/agent_loop/slowest/response_length:3017 - timing_s/gen:27.816161528229713 - timing_s/reward:0.0652250312268734 - timing_s/old_log_prob:3.518872758373618 - timing_s/adv:0.15252801775932312 - timing_s/update_actor:13.777937777340412 - timing_s/step:45.40992152877152 - timing_s/stop_profile:0.00012294389307498932 - timing_per_token_ms/gen:0.16935979937184364 - timing_per_token_ms/update_actor:0.059170618882205414 - timing_per_token_ms/adv:0.0006550455774693822 - perf/total_num_tokens:232851 - perf/time_per_step:45.40992152877152 - perf/throughput:640.9695066651532 (TaskRunner pid=2281424) Training Progress: 21%|██ | 21/100 [20:09<1:43:38, 78.71s/it] (TaskRunner pid=2281424) step:22 - global_seqlen/min:20123 - global_seqlen/max:45886 - global_seqlen/minmax_diff:25763 - global_seqlen/balanced_min:32892 - global_seqlen/balanced_max:32946 - global_seqlen/mean:32919.625 - actor/entropy:0.2954756021499634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5972118377685547 - training/rollout_probs_diff_mean:0.0029836043249815702 - training/rollout_probs_diff_std:0.008759740740060806 - training/rollout_actor_probs_pearson_corr:0.9994177222251892 - rollout_corr/rollout_is_mean:1.000027060508728 - rollout_corr/rollout_is_ratio_fraction_high:1.0150790330953896e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5226184586936142e-05 - rollout_corr/rollout_is_max:5.339309215545654 - rollout_corr/rollout_is_min:0.3289409577846527 - rollout_corr/rollout_is_std:0.027539236471056938 - rollout_corr/rollout_is_eff_sample_size:0.999242284277643 - rollout_corr/rollout_is_seq_mean:1.000131607055664 - rollout_corr/rollout_is_seq_std:0.0019331261282786727 - rollout_corr/rollout_is_seq_max:1.0212081670761108 - rollout_corr/rollout_is_seq_min:0.990929365158081 - rollout_corr/rollout_is_seq_max_deviation:0.02120816707611084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2931092074140906) - rollout_corr/training_log_ppl:np.float64(0.24355409591225907) - rollout_corr/kl:np.float64(0.0005864815489502462) - rollout_corr/k3_kl:np.float64(0.0005580453379678829) - rollout_corr/rollout_ppl:np.float64(1.292321430053562) - rollout_corr/rollout_log_ppl:np.float64(0.24296761279401835) - rollout_corr/log_ppl_diff:np.float64(0.0005864831182407215) - rollout_corr/log_ppl_abs_diff:np.float64(0.001289499065023847) - rollout_corr/log_ppl_diff_max:np.float64(0.019712358713150024) - rollout_corr/log_ppl_diff_min:np.float64(-0.007148668169975281) - rollout_corr/ppl_ratio:np.float64(1.0005892012268305) - rollout_corr/chi2_token:np.float64(0.0012211541179567575) - rollout_corr/chi2_seq:np.float64(0.6782732140272856) - actor/pg_loss:np.float64(0.00016800372395664454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002511092553731942) - actor/ppo_kl:np.float64(0.00027240615360524245) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29514269530773163) - perf/mfu/actor:np.float64(0.12364022737507745) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.99796676635742) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005714894738048315 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005714894738048315 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:769.64453125 - response_length/max:2870.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:769.64453125 - response_length_non_aborted/max:2870.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.717978537082672e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7690473347902298) - timing_s/agent_loop/generate_sequences/max:np.float64(21.679362626746297) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.051158084097551) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.679362626746297) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:2870 - timing_s/gen:28.594687478616834 - timing_s/reward:0.07660721056163311 - timing_s/old_log_prob:3.3768766596913338 - timing_s/adv:0.14736606925725937 - timing_s/update_actor:14.125624496489763 - timing_s/step:46.4090982247144 - timing_s/stop_profile:0.00013731606304645538 - timing_per_token_ms/gen:0.1451293336443713 - timing_per_token_ms/update_actor:0.0536367914902196 - timing_per_token_ms/adv:0.0005595676942601084 - perf/total_num_tokens:263357 - perf/time_per_step:46.4090982247144 - perf/throughput:709.3355884788383 (TaskRunner pid=2281424) Training Progress: 22%|██▏ | 22/100 [20:55<1:29:44, 69.04s/it] (TaskRunner pid=2281424) step:23 - global_seqlen/min:22934 - global_seqlen/max:53809 - global_seqlen/minmax_diff:30875 - global_seqlen/balanced_min:39243 - global_seqlen/balanced_max:39551 - global_seqlen/mean:39306.5 - actor/entropy:0.33397042751312256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4463748633861542 - training/rollout_probs_diff_mean:0.003143494250252843 - training/rollout_probs_diff_std:0.008556975051760674 - training/rollout_actor_probs_pearson_corr:0.9994155764579773 - rollout_corr/rollout_is_mean:0.9999798536300659 - rollout_corr/rollout_is_ratio_fraction_high:8.167132364178542e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6334264728357084e-05 - rollout_corr/rollout_is_max:2.7373318672180176 - rollout_corr/rollout_is_min:0.23669613897800446 - rollout_corr/rollout_is_std:0.028452597558498383 - rollout_corr/rollout_is_eff_sample_size:0.9991909855397003 - rollout_corr/rollout_is_seq_mean:0.9999604821205139 - rollout_corr/rollout_is_seq_std:0.001166414120234549 - rollout_corr/rollout_is_seq_max:1.0046206712722778 - rollout_corr/rollout_is_seq_min:0.9961167573928833 - rollout_corr/rollout_is_seq_max_deviation:0.004620671272277832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.385243427939713) - rollout_corr/training_log_ppl:np.float64(0.3101144570391625) - rollout_corr/kl:np.float64(0.0005338342346767888) - rollout_corr/k3_kl:np.float64(0.000465180856878078) - rollout_corr/rollout_ppl:np.float64(1.3844622108153999) - rollout_corr/rollout_log_ppl:np.float64(0.30958062058198266) - rollout_corr/log_ppl_diff:np.float64(0.0005338364571798593) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009809565672185272) - rollout_corr/log_ppl_diff_max:np.float64(0.006667211651802063) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032280683517456055) - rollout_corr/ppl_ratio:np.float64(1.0005347987171263) - rollout_corr/chi2_token:np.float64(0.000802481546998024) - rollout_corr/chi2_seq:np.float64(0.3838173961266875) - actor/pg_loss:np.float64(6.97027426213026e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030751674410112173) - actor/ppo_kl:np.float64(6.243631206404032e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20673737302422523) - perf/mfu/actor:np.float64(0.1370845535950862) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(105.6169319152832) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0067174662835896015 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0067174662835896015 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:956.578125 - response_length/max:4024.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:956.578125 - response_length_non_aborted/max:4024.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00028231181204319 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8497151639312506) - timing_s/agent_loop/generate_sequences/max:np.float64(29.862245429307222) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.941006362365442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.862245429307222) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:4024 - timing_s/gen:36.85486473329365 - timing_s/reward:0.08099697157740593 - timing_s/old_log_prob:3.847181210294366 - timing_s/adv:0.1430294755846262 - timing_s/update_actor:15.214192001149058 - timing_s/step:56.236885111778975 - timing_s/stop_profile:0.00012627430260181427 - timing_per_token_ms/gen:0.1504992761196879 - timing_per_token_ms/update_actor:0.04838319362303009 - timing_per_token_ms/adv:0.00045485312729645925 - perf/total_num_tokens:314452 - perf/time_per_step:56.236885111778975 - perf/throughput:698.9451837859195 (TaskRunner pid=2281424) Training Progress: 23%|██▎ | 23/100 [21:52<1:23:49, 65.31s/it] (TaskRunner pid=2281424) step:24 - global_seqlen/min:21247 - global_seqlen/max:40900 - global_seqlen/minmax_diff:19653 - global_seqlen/balanced_min:30630 - global_seqlen/balanced_max:32071 - global_seqlen/mean:30857.125 - actor/entropy:0.31415796279907227 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3094596862792969 - training/rollout_probs_diff_mean:0.003211991162970662 - training/rollout_probs_diff_std:0.008898185566067696 - training/rollout_actor_probs_pearson_corr:0.9993066191673279 - rollout_corr/rollout_is_mean:0.9999200105667114 - rollout_corr/rollout_is_ratio_fraction_high:5.59005866307416e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.59005866307416e-06 - rollout_corr/rollout_is_max:2.404808521270752 - rollout_corr/rollout_is_min:0.19617219269275665 - rollout_corr/rollout_is_std:0.02825077250599861 - rollout_corr/rollout_is_eff_sample_size:0.9992022922335758 - rollout_corr/rollout_is_seq_mean:0.9998365640640259 - rollout_corr/rollout_is_seq_std:0.001564831123687327 - rollout_corr/rollout_is_seq_max:1.0052552223205566 - rollout_corr/rollout_is_seq_min:0.992292582988739 - rollout_corr/rollout_is_seq_max_deviation:0.007707417011260986 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.303541206754744) - rollout_corr/training_log_ppl:np.float64(0.25006278717773966) - rollout_corr/kl:np.float64(0.0005476736968059726) - rollout_corr/k3_kl:np.float64(0.00043206283888252983) - rollout_corr/rollout_ppl:np.float64(1.3028357764706016) - rollout_corr/rollout_log_ppl:np.float64(0.24951511321705766) - rollout_corr/log_ppl_diff:np.float64(0.0005476739606820047) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011344742961227894) - rollout_corr/log_ppl_diff_max:np.float64(0.008412361145019531) - rollout_corr/log_ppl_diff_min:np.float64(-0.00302673876285553) - rollout_corr/ppl_ratio:np.float64(1.0005489422474056) - rollout_corr/chi2_token:np.float64(0.0006379920523613691) - rollout_corr/chi2_seq:np.float64(0.4201601871754974) - actor/pg_loss:np.float64(-3.0018971301615238e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.42312039076387e-05) - actor/ppo_kl:np.float64(-2.9987399832176465e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1152929738163948) - perf/mfu/actor:np.float64(0.11669484239312708) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.80078125) - perf/cpu_memory_used_gb:np.float64(105.66312026977539) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013332289643585682 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013332289643585682 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:698.78515625 - response_length/max:4217.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:698.78515625 - response_length_non_aborted/max:4217.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001115109771490097 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1027332954108715) - timing_s/agent_loop/generate_sequences/max:np.float64(28.570936573669314) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.08747353394574) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.570936573669314) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:4217 - timing_s/gen:35.1314688809216 - timing_s/reward:0.07224285788834095 - timing_s/old_log_prob:3.3607450742274523 - timing_s/adv:0.1696465015411377 - timing_s/update_actor:13.863193551078439 - timing_s/step:52.68583432585001 - timing_s/stop_profile:0.00012148357927799225 - timing_per_token_ms/gen:0.19638697114367906 - timing_per_token_ms/update_actor:0.05615880267149985 - timing_per_token_ms/adv:0.0006872258090357482 - perf/total_num_tokens:246857 - perf/time_per_step:52.68583432585001 - perf/throughput:585.6816238147742 (TaskRunner pid=2281424) Training Progress: 24%|██▍ | 24/100 [22:45<1:17:57, 61.54s/it] (TaskRunner pid=2281424) step:25 - global_seqlen/min:23246 - global_seqlen/max:52171 - global_seqlen/minmax_diff:28925 - global_seqlen/balanced_min:34912 - global_seqlen/balanced_max:36421 - global_seqlen/mean:35164.75 - actor/entropy:0.35677048563957214 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5146784782409668 - training/rollout_probs_diff_mean:0.0033477256074547768 - training/rollout_probs_diff_std:0.00895012728869915 - training/rollout_actor_probs_pearson_corr:0.9994227886199951 - rollout_corr/rollout_is_mean:0.9999942779541016 - rollout_corr/rollout_is_ratio_fraction_high:1.4294563698058482e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4294563698058482e-05 - rollout_corr/rollout_is_max:2.550842046737671 - rollout_corr/rollout_is_min:0.2724951505661011 - rollout_corr/rollout_is_std:0.02962060645222664 - rollout_corr/rollout_is_eff_sample_size:0.9991233887504097 - rollout_corr/rollout_is_seq_mean:1.0000044107437134 - rollout_corr/rollout_is_seq_std:0.001464587403461337 - rollout_corr/rollout_is_seq_max:1.0044803619384766 - rollout_corr/rollout_is_seq_min:0.9913833737373352 - rollout_corr/rollout_is_seq_max_deviation:0.008616626262664795 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3856255286373198) - rollout_corr/training_log_ppl:np.float64(0.3002330707240617) - rollout_corr/kl:np.float64(0.00044336768771691126) - rollout_corr/k3_kl:np.float64(0.0005020772942430085) - rollout_corr/rollout_ppl:np.float64(1.3849827772937715) - rollout_corr/rollout_log_ppl:np.float64(0.2997897020104574) - rollout_corr/log_ppl_diff:np.float64(0.0004433687136042863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010328854841645807) - rollout_corr/log_ppl_diff_max:np.float64(0.010214567184448242) - rollout_corr/log_ppl_diff_min:np.float64(-0.005682602524757385) - rollout_corr/ppl_ratio:np.float64(1.000444594072178) - rollout_corr/chi2_token:np.float64(0.0011366589460521936) - rollout_corr/chi2_seq:np.float64(0.17277534841559827) - actor/pg_loss:np.float64(-0.00010206212755292654) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003034779521726705) - actor/ppo_kl:np.float64(-3.511990449567293e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1835123896598816) - perf/mfu/actor:np.float64(0.1283748813468745) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.58767604827881) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.025912469252943993 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.025912469252943993 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:819.8046875 - response_length/max:5123.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:819.8046875 - response_length_non_aborted/max:5123.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001536533236503601 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2272972408682108) - timing_s/agent_loop/generate_sequences/max:np.float64(33.647482462227345) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.377409354601696) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.647482462227345) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:5123 - timing_s/gen:40.239728681743145 - timing_s/reward:0.07882356271147728 - timing_s/old_log_prob:3.5549416448920965 - timing_s/adv:0.13953095860779285 - timing_s/update_actor:14.543323775753379 - timing_s/step:58.64243852160871 - timing_s/stop_profile:0.00014534033834934235 - timing_per_token_ms/gen:0.1917364496199702 - timing_per_token_ms/update_actor:0.05169709643802878 - timing_per_token_ms/adv:0.0004959901556522969 - perf/total_num_tokens:281318 - perf/time_per_step:58.64243852160871 - perf/throughput:599.646789705759 (TaskRunner pid=2281424) Training Progress: 25%|██▌ | 25/100 [23:43<1:15:51, 60.69s/it] (TaskRunner pid=2281424) step:26 - global_seqlen/min:24712 - global_seqlen/max:39123 - global_seqlen/minmax_diff:14411 - global_seqlen/balanced_min:30297 - global_seqlen/balanced_max:30365 - global_seqlen/mean:30344.375 - actor/entropy:0.2876966595649719 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20822957158088684 - training/rollout_probs_diff_mean:0.0029764524661004543 - training/rollout_probs_diff_std:0.008548058569431305 - training/rollout_actor_probs_pearson_corr:0.9995021820068359 - rollout_corr/rollout_is_mean:1.0000873804092407 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9248120784759521 - rollout_corr/rollout_is_min:0.5073025822639465 - rollout_corr/rollout_is_std:0.02774408645927906 - rollout_corr/rollout_is_eff_sample_size:0.9992309766786769 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.0015661150682717562 - rollout_corr/rollout_is_seq_max:1.0046665668487549 - rollout_corr/rollout_is_seq_min:0.9934303760528564 - rollout_corr/rollout_is_seq_max_deviation:0.006569623947143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3186388644389808) - rollout_corr/training_log_ppl:np.float64(0.2604680637450656) - rollout_corr/kl:np.float64(0.0005275833006166408) - rollout_corr/k3_kl:np.float64(0.0005207556534685409) - rollout_corr/rollout_ppl:np.float64(1.3179062088020146) - rollout_corr/rollout_log_ppl:np.float64(0.259940478586941) - rollout_corr/log_ppl_diff:np.float64(0.0005275851581245661) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012775544892065227) - rollout_corr/log_ppl_diff_max:np.float64(0.009042024612426758) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037787258625030518) - rollout_corr/ppl_ratio:np.float64(1.0005293833091855) - rollout_corr/chi2_token:np.float64(0.0010106251575052738) - rollout_corr/chi2_seq:np.float64(1.5079653931315988) - actor/pg_loss:np.float64(0.00019484246149659157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004149369976857997) - actor/ppo_kl:np.float64(0.00010909117361990184) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17218420654535294) - perf/mfu/actor:np.float64(0.11675351697015622) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.56155776977539) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007410742342472076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007410742342472076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:688.73046875 - response_length/max:2773.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:688.73046875 - response_length_non_aborted/max:2773.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011996924877166748 - timing_s/agent_loop/generate_sequences/min:np.float64(1.479305861517787) - timing_s/agent_loop/generate_sequences/max:np.float64(20.65613953024149) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.7965023621873115) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.65613953024149) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:2773 - timing_s/gen:27.100659066811204 - timing_s/reward:0.07199572958052158 - timing_s/old_log_prob:3.321561750024557 - timing_s/adv:0.16026310436427593 - timing_s/update_actor:13.673425955697894 - timing_s/step:44.414213774725795 - timing_s/stop_profile:0.00011827610433101654 - timing_per_token_ms/gen:0.1537059187636401 - timing_per_token_ms/update_actor:0.05632603223702043 - timing_per_token_ms/adv:0.0006601845661851494 - perf/total_num_tokens:242755 - perf/time_per_step:44.414213774725795 - perf/throughput:683.2131523009796 (TaskRunner pid=2281424) Training Progress: 26%|██▌ | 26/100 [24:28<1:08:50, 55.82s/it] (TaskRunner pid=2281424) step:27 - global_seqlen/min:25977 - global_seqlen/max:48183 - global_seqlen/minmax_diff:22206 - global_seqlen/balanced_min:36093 - global_seqlen/balanced_max:36153 - global_seqlen/mean:36121.125 - actor/entropy:0.30439427495002747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2310372292995453 - training/rollout_probs_diff_mean:0.002941888989880681 - training/rollout_probs_diff_std:0.008349236100912094 - training/rollout_actor_probs_pearson_corr:0.99952632188797 - rollout_corr/rollout_is_mean:0.9999545216560364 - rollout_corr/rollout_is_ratio_fraction_high:4.5820902414561715e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3746271179115865e-05 - rollout_corr/rollout_is_max:2.7575740814208984 - rollout_corr/rollout_is_min:0.4973885416984558 - rollout_corr/rollout_is_std:0.027530577033758163 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:0.9999428987503052 - rollout_corr/rollout_is_seq_std:0.0012769426684826612 - rollout_corr/rollout_is_seq_max:1.004395604133606 - rollout_corr/rollout_is_seq_min:0.9921810030937195 - rollout_corr/rollout_is_seq_max_deviation:0.007818996906280518 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3105780305340886) - rollout_corr/training_log_ppl:np.float64(0.25283126058639027) - rollout_corr/kl:np.float64(0.00047030424940075477) - rollout_corr/k3_kl:np.float64(0.0004522940477897919) - rollout_corr/rollout_ppl:np.float64(1.3099122606217861) - rollout_corr/rollout_log_ppl:np.float64(0.2523609542840859) - rollout_corr/log_ppl_diff:np.float64(0.00047030630230437964) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010534748871577904) - rollout_corr/log_ppl_diff_max:np.float64(0.009107545018196106) - rollout_corr/log_ppl_diff_min:np.float64(-0.00359538197517395) - rollout_corr/ppl_ratio:np.float64(1.0004714394453913) - rollout_corr/chi2_token:np.float64(0.0008890700992196798) - rollout_corr/chi2_seq:np.float64(1.7405338953249156) - actor/pg_loss:np.float64(-9.15073323994875e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002051412569699096) - actor/ppo_kl:np.float64(1.4041290314459332e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15001486241817474) - perf/mfu/actor:np.float64(0.13349702138291186) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.56867599487305) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006906355731189251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006906355731189251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:852.50390625 - response_length/max:3307.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:852.50390625 - response_length_non_aborted/max:3307.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015197880566120148 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5302176997065544) - timing_s/agent_loop/generate_sequences/max:np.float64(24.74932035431266) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.707113990400103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.74932035431266) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:3281 - timing_s/gen:31.510395139455795 - timing_s/reward:0.07958226092159748 - timing_s/old_log_prob:3.521709840744734 - timing_s/adv:0.14874472096562386 - timing_s/update_actor:14.358893560245633 - timing_s/step:49.70740628428757 - timing_s/stop_profile:0.00017487630248069763 - timing_per_token_ms/gen:0.144383480370122 - timing_per_token_ms/update_actor:0.049690082881712686 - timing_per_token_ms/adv:0.0005147428304268757 - perf/total_num_tokens:288969 - perf/time_per_step:49.70740628428757 - perf/throughput:726.6749102420544 (TaskRunner pid=2281424) Training Progress: 27%|██▋ | 27/100 [25:18<1:05:42, 54.01s/it] (TaskRunner pid=2281424) step:28 - global_seqlen/min:21975 - global_seqlen/max:58966 - global_seqlen/minmax_diff:36991 - global_seqlen/balanced_min:38635 - global_seqlen/balanced_max:38827 - global_seqlen/mean:38769.375 - actor/entropy:0.3461316227912903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34711864590644836 - training/rollout_probs_diff_mean:0.003255465067923069 - training/rollout_probs_diff_std:0.008680691011250019 - training/rollout_actor_probs_pearson_corr:0.9994996190071106 - rollout_corr/rollout_is_mean:1.00006103515625 - rollout_corr/rollout_is_ratio_fraction_high:8.337119652424008e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.084279913106002e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.17730121314525604 - rollout_corr/rollout_is_std:0.028798246756196022 - rollout_corr/rollout_is_eff_sample_size:0.9991714672176878 - rollout_corr/rollout_is_seq_mean:1.0000207424163818 - rollout_corr/rollout_is_seq_std:0.0014608351048082113 - rollout_corr/rollout_is_seq_max:1.006380558013916 - rollout_corr/rollout_is_seq_min:0.9955966472625732 - rollout_corr/rollout_is_seq_max_deviation:0.006380558013916016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3715200861915946) - rollout_corr/training_log_ppl:np.float64(0.2959491069341311) - rollout_corr/kl:np.float64(0.0004920310259990401) - rollout_corr/k3_kl:np.float64(0.000492703045580356) - rollout_corr/rollout_ppl:np.float64(1.3707749480381608) - rollout_corr/rollout_log_ppl:np.float64(0.2954570743313525) - rollout_corr/log_ppl_diff:np.float64(0.0004920326027786359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011141302093164995) - rollout_corr/log_ppl_diff_max:np.float64(0.00506967306137085) - rollout_corr/log_ppl_diff_min:np.float64(-0.004320159554481506) - rollout_corr/ppl_ratio:np.float64(1.0004931495059282) - rollout_corr/chi2_token:np.float64(0.0010096586775034666) - rollout_corr/chi2_seq:np.float64(1.0986658702604473) - actor/pg_loss:np.float64(-3.765069413930178e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001984241041554924) - actor/ppo_kl:np.float64(4.411382598945579e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1757737323641777) - perf/mfu/actor:np.float64(0.13821070890339499) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.6130256652832) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01265887450426817 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01265887450426817 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:937.07421875 - response_length/max:4442.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:937.07421875 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013406015932559967 - timing_s/agent_loop/generate_sequences/min:np.float64(1.040765456855297) - timing_s/agent_loop/generate_sequences/max:np.float64(31.025565603747964) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.124727715730842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(31.025565603747964) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:4440 - timing_s/gen:37.81342346780002 - timing_s/reward:0.08299151435494423 - timing_s/old_log_prob:3.631516432389617 - timing_s/adv:0.1339629665017128 - timing_s/update_actor:15.046643575653434 - timing_s/step:56.79524556733668 - timing_s/stop_profile:0.00011165253818035126 - timing_per_token_ms/gen:0.1576275202812945 - timing_per_token_ms/update_actor:0.04851330326982778 - timing_per_token_ms/adv:0.00043192264029827924 - perf/total_num_tokens:310155 - perf/time_per_step:56.79524556733668 - perf/throughput:682.61655729677 (TaskRunner pid=2281424) Training Progress: 28%|██▊ | 28/100 [26:14<1:05:49, 54.86s/it] (TaskRunner pid=2281424) step:29 - global_seqlen/min:26468 - global_seqlen/max:47121 - global_seqlen/minmax_diff:20653 - global_seqlen/balanced_min:34309 - global_seqlen/balanced_max:34448 - global_seqlen/mean:34354.375 - actor/entropy:0.2958447337150574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135512113571167 - training/rollout_probs_diff_mean:0.003082486568018794 - training/rollout_probs_diff_std:0.00864423532038927 - training/rollout_actor_probs_pearson_corr:0.9994667768478394 - rollout_corr/rollout_is_mean:0.9999562501907349 - rollout_corr/rollout_is_ratio_fraction_high:4.822275059268577e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.446682472305838e-05 - rollout_corr/rollout_is_max:2.0740530490875244 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.027990059927105904 - rollout_corr/rollout_is_eff_sample_size:0.9992171698522248 - rollout_corr/rollout_is_seq_mean:0.9999083280563354 - rollout_corr/rollout_is_seq_std:0.0016167733119800687 - rollout_corr/rollout_is_seq_max:1.008034586906433 - rollout_corr/rollout_is_seq_min:0.993806004524231 - rollout_corr/rollout_is_seq_max_deviation:0.008034586906433105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3131270464509726) - rollout_corr/training_log_ppl:np.float64(0.26043001232028473) - rollout_corr/kl:np.float64(0.0006687698600753578) - rollout_corr/k3_kl:np.float64(0.0005304543523152461) - rollout_corr/rollout_ppl:np.float64(1.3122303052805364) - rollout_corr/rollout_log_ppl:np.float64(0.2597612422541715) - rollout_corr/log_ppl_diff:np.float64(0.0006687700661132112) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013376595597947016) - rollout_corr/log_ppl_diff_max:np.float64(0.009146973490715027) - rollout_corr/log_ppl_diff_min:np.float64(-0.004970654845237732) - rollout_corr/ppl_ratio:np.float64(1.0006705992855132) - rollout_corr/chi2_token:np.float64(0.0007638768292963505) - rollout_corr/chi2_seq:np.float64(0.7327017430216074) - actor/pg_loss:np.float64(4.302803426980972e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002477214987948173) - actor/ppo_kl:np.float64(0.00011317849087255638) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1297485176473856) - perf/mfu/actor:np.float64(0.1281224223428748) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.53421020507812) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04238719493150711 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04238719493150711 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:810.04296875 - response_length/max:3813.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:810.04296875 - response_length_non_aborted/max:3813.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014430470764636993 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3452266938984394) - timing_s/agent_loop/generate_sequences/max:np.float64(27.088402163237333) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.822816854313714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.088402163237333) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3813 - timing_s/gen:33.822356663644314 - timing_s/reward:0.073737021535635 - timing_s/old_log_prob:3.5272585432976484 - timing_s/adv:0.16648594662547112 - timing_s/update_actor:14.246411005035043 - timing_s/step:51.92763191089034 - timing_s/stop_profile:6.692670285701752e-05 - timing_per_token_ms/gen:0.16310070677020563 - timing_per_token_ms/update_actor:0.051836232667000354 - timing_per_token_ms/adv:0.0006057669024158899 - perf/total_num_tokens:274835 - perf/time_per_step:51.92763191089034 - perf/throughput:661.5817770961196 (TaskRunner pid=2281424) Training Progress: 29%|██▉ | 29/100 [27:06<1:03:53, 53.99s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We can solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substituting the values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \cdot (0.50 \, \mathrm{m})^2}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0) \cdot (0.25)}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, convert coulombs to picocoulombs (1 pC = $1 \times 10^{-12} \, \mathrm{C}$): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2281424) step:30 - global_seqlen/min:21152 - global_seqlen/max:43378 - global_seqlen/minmax_diff:22226 - global_seqlen/balanced_min:32690 - global_seqlen/balanced_max:36508 - global_seqlen/mean:33253.625 - actor/entropy:0.3238883912563324 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535161018371582 - training/rollout_probs_diff_mean:0.0032308220397681 - training/rollout_probs_diff_std:0.00875203125178814 - training/rollout_actor_probs_pearson_corr:0.9993578195571899 - rollout_corr/rollout_is_mean:1.0000488758087158 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9363911151885986 - rollout_corr/rollout_is_min:0.5356025099754333 - rollout_corr/rollout_is_std:0.028555061668157578 - rollout_corr/rollout_is_eff_sample_size:0.9991853917962915 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.0014182842569425702 - rollout_corr/rollout_is_seq_max:1.00834321975708 - rollout_corr/rollout_is_seq_min:0.9960029721260071 - rollout_corr/rollout_is_seq_max_deviation:0.008343219757080078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3260084195062518) - rollout_corr/training_log_ppl:np.float64(0.26798260471878166) - rollout_corr/kl:np.float64(0.00034142893949073994) - rollout_corr/k3_kl:np.float64(0.00046080480333898777) - rollout_corr/rollout_ppl:np.float64(1.3255186867900193) - rollout_corr/rollout_log_ppl:np.float64(0.2676411745214864) - rollout_corr/log_ppl_diff:np.float64(0.00034143019729526713) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010471087371115573) - rollout_corr/log_ppl_diff_max:np.float64(0.0044901371002197266) - rollout_corr/log_ppl_diff_min:np.float64(-0.006970256567001343) - rollout_corr/ppl_ratio:np.float64(1.0003424619790167) - rollout_corr/chi2_token:np.float64(0.0011780557688325644) - rollout_corr/chi2_seq:np.float64(9.059563745046034) - actor/pg_loss:np.float64(-7.204711437225342e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002318803169032435) - actor/ppo_kl:np.float64(-2.2715349961899278e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1330014280974865) - perf/mfu/actor:np.float64(0.11964730091498024) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.54463195800781) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6046875) - val-aux/sciknoweval/reward/std@16:np.float64(0.19331331768802099) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6854) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1699013098775457) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5269125000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1573208332576977) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6056250000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19436455511085113) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.75095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13208485427187178) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4644124999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11165691249499834) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6343625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16176855495856937) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8039375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0971781630054231) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42205000000000004) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07114364007214655) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6537749999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12550218875311617) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.84275) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061103157160745235) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.39563750000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.038223937353486045) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.666125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09165205807525627) - val-aux/sciknoweval/score/mean@16:np.float64(0.6046875) - val-aux/sciknoweval/score/std@16:np.float64(0.19331331768802099) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6854) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1699013098775457) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5269125000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1573208332576977) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6056250000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19436455511085113) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.75095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13208485427187178) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4644124999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11165691249499834) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6343625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16176855495856937) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8039375) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0971781630054231) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42205000000000004) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07114364007214655) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6537749999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12550218875311617) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.84275) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061103157160745235) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.39563750000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.038223937353486045) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.666125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09165205807525627) - val-core/sciknoweval/acc/mean@16:np.float64(0.6046875) - val-aux/sciknoweval/acc/std@16:np.float64(0.19331331768802099) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6854) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1699013098775457) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5269125000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1573208332576977) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6056250000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19436455511085113) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.75095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13208485427187178) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4644124999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11165691249499834) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6343625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16176855495856937) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8039375) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0971781630054231) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42205000000000004) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07114364007214655) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6537749999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12550218875311617) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.84275) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061103157160745235) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.39563750000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.038223937353486045) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.666125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09165205807525627) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9421875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.11810324679239408) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9847249999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05478145372165707) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9000999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.14152552996328321) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9422875000000002) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11789115899569932) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9971625) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.012950626299582524) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8410875000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.15275603349423345) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.965475) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.08685302210496705) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0022688782927956746) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7717375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.1420235663973301) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9818749999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.05503683300798294) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.70755) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.10363469468730849) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9918625000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.02568992930006094) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00041142108966596425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00041142108966596425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:779.73828125 - response_length/max:8192.0 - response_length/min:130.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:779.73828125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012162141501903534 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5774449706077576) - timing_s/agent_loop/generate_sequences/max:np.float64(62.31554364040494) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.508535967179341) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(62.31554364040494) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:323 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:69.08000029809773 - timing_s/reward:0.07287852838635445 - timing_s/old_log_prob:3.7097500916570425 - timing_s/adv:0.15018161572515965 - timing_s/update_actor:14.847722072154284 - timing_s/step:87.9640430957079 - timing_s/testing:149.87052939273417 - timing_s/save_checkpoint:12.872908538207412 - timing_s/stop_profile:0.00015308894217014313 - timing_per_token_ms/gen:0.346069646255994 - timing_per_token_ms/update_actor:0.05581241921803368 - timing_per_token_ms/adv:0.0005645309937080531 - perf/total_num_tokens:266029 - perf/time_per_step:87.9640430957079 - perf/throughput:378.03656846262646 (TaskRunner pid=2281424) Training Progress: 30%|███ | 30/100 [31:17<2:11:52, 113.03s/it] (TaskRunner pid=2281424) step:31 - global_seqlen/min:26976 - global_seqlen/max:51365 - global_seqlen/minmax_diff:24389 - global_seqlen/balanced_min:37711 - global_seqlen/balanced_max:37821 - global_seqlen/mean:37765.0 - actor/entropy:0.33241793513298035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.301638126373291 - training/rollout_probs_diff_mean:0.003111757803708315 - training/rollout_probs_diff_std:0.008627519011497498 - training/rollout_actor_probs_pearson_corr:0.9994876980781555 - rollout_corr/rollout_is_mean:0.9999532699584961 - rollout_corr/rollout_is_ratio_fraction_high:8.658907972858287e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.2988362868782133e-05 - rollout_corr/rollout_is_max:2.167019844055176 - rollout_corr/rollout_is_min:0.3715561032295227 - rollout_corr/rollout_is_std:0.028119659051299095 - rollout_corr/rollout_is_eff_sample_size:0.9992096714770495 - rollout_corr/rollout_is_seq_mean:0.9999525547027588 - rollout_corr/rollout_is_seq_std:0.0012215663446113467 - rollout_corr/rollout_is_seq_max:1.004585862159729 - rollout_corr/rollout_is_seq_min:0.9942855834960938 - rollout_corr/rollout_is_seq_max_deviation:0.00571441650390625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3133017774671316) - rollout_corr/training_log_ppl:np.float64(0.25711654864426237) - rollout_corr/kl:np.float64(0.0004556128830559558) - rollout_corr/k3_kl:np.float64(0.00039641217307462284) - rollout_corr/rollout_ppl:np.float64(1.3126854803413153) - rollout_corr/rollout_log_ppl:np.float64(0.25666093576001003) - rollout_corr/log_ppl_diff:np.float64(0.00045561288425233215) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010043695365311578) - rollout_corr/log_ppl_diff_max:np.float64(0.006338976323604584) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035679787397384644) - rollout_corr/ppl_ratio:np.float64(1.0004564973060042) - rollout_corr/chi2_token:np.float64(0.0006840897258371115) - rollout_corr/chi2_seq:np.float64(0.9153507652226835) - actor/pg_loss:np.float64(1.1453521437942982e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001992799257095612) - actor/ppo_kl:np.float64(2.699381930071354e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14760523661971092) - perf/mfu/actor:np.float64(0.13769359300581646) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.38329315185547) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009976144880056381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009976144880056381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:902.25 - response_length/max:3456.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:902.25 - response_length_non_aborted/max:3456.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019257515668869019 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1455713659524918) - timing_s/agent_loop/generate_sequences/max:np.float64(25.956687213853) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.312753760583291) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.956687213853) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:3456 - timing_s/gen:32.13823664188385 - timing_s/reward:0.0789014957845211 - timing_s/old_log_prob:3.81612709723413 - timing_s/adv:0.13714739680290222 - timing_s/update_actor:14.64492060430348 - timing_s/step:50.91287592239678 - timing_s/stop_profile:0.00011721253395080566 - timing_per_token_ms/gen:0.13914102175933366 - timing_per_token_ms/update_actor:0.048473853449965174 - timing_per_token_ms/adv:0.00045395007547630815 - perf/total_num_tokens:302120 - perf/time_per_step:50.91287592239678 - perf/throughput:741.7573514716152 (TaskRunner pid=2281424) Training Progress: 31%|███ | 31/100 [32:08<1:48:34, 94.42s/it] (TaskRunner pid=2281424) step:32 - global_seqlen/min:17749 - global_seqlen/max:40522 - global_seqlen/minmax_diff:22773 - global_seqlen/balanced_min:29749 - global_seqlen/balanced_max:30114 - global_seqlen/mean:29860.25 - actor/entropy:0.3001790940761566 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931319773197174 - training/rollout_probs_diff_mean:0.0032425629906356335 - training/rollout_probs_diff_std:0.009085338562726974 - training/rollout_actor_probs_pearson_corr:0.9993113279342651 - rollout_corr/rollout_is_mean:1.0000972747802734 - rollout_corr/rollout_is_ratio_fraction_high:5.759240593761206e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.3036962375044823e-05 - rollout_corr/rollout_is_max:2.2561657428741455 - rollout_corr/rollout_is_min:0.11440415680408478 - rollout_corr/rollout_is_std:0.028905672952532768 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0000715255737305 - rollout_corr/rollout_is_seq_std:0.001591670559719205 - rollout_corr/rollout_is_seq_max:1.0057041645050049 - rollout_corr/rollout_is_seq_min:0.9924244284629822 - rollout_corr/rollout_is_seq_max_deviation:0.007575571537017822 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3085072333924472) - rollout_corr/training_log_ppl:np.float64(0.25189579716243315) - rollout_corr/kl:np.float64(0.0004256223451228891) - rollout_corr/k3_kl:np.float64(0.0005207587888378384) - rollout_corr/rollout_ppl:np.float64(1.307915877085179) - rollout_corr/rollout_log_ppl:np.float64(0.25147017439303454) - rollout_corr/log_ppl_diff:np.float64(0.00042562276939861476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012707264686468989) - rollout_corr/log_ppl_diff_max:np.float64(0.008575737476348877) - rollout_corr/log_ppl_diff_min:np.float64(-0.005954384803771973) - rollout_corr/ppl_ratio:np.float64(1.0004271555226296) - rollout_corr/chi2_token:np.float64(0.0012784942518919706) - rollout_corr/chi2_seq:np.float64(1.3941792568657547) - actor/pg_loss:np.float64(-4.464993253350258e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011340672926962725) - actor/ppo_kl:np.float64(3.421145366999667e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10089067462831736) - perf/mfu/actor:np.float64(0.11507569937509035) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.87258911132812) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02760116197168827 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02760116197168827 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:678.2578125 - response_length/max:3951.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:678.2578125 - response_length_non_aborted/max:3951.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010407157242298126 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1292062569409609) - timing_s/agent_loop/generate_sequences/max:np.float64(27.004830926656723) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.73606214641768) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.004830926656723) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:3951 - timing_s/gen:34.06722195446491 - timing_s/reward:0.06839162670075893 - timing_s/old_log_prob:3.350517366081476 - timing_s/adv:0.15078148432075977 - timing_s/update_actor:13.741796219721437 - timing_s/step:51.4739043507725 - timing_s/stop_profile:0.00011109933257102966 - timing_per_token_ms/gen:0.1962013312742027 - timing_per_token_ms/update_actor:0.05752545700271028 - timing_per_token_ms/adv:0.0006311965084048181 - perf/total_num_tokens:238882 - perf/time_per_step:51.4739043507725 - perf/throughput:580.104625375127 (TaskRunner pid=2281424) Training Progress: 32%|███▏ | 32/100 [33:00<1:32:25, 81.55s/it] (TaskRunner pid=2281424) step:33 - global_seqlen/min:27211 - global_seqlen/max:50719 - global_seqlen/minmax_diff:23508 - global_seqlen/balanced_min:36245 - global_seqlen/balanced_max:36302 - global_seqlen/mean:36278.125 - actor/entropy:0.30930498242378235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351723432540894 - training/rollout_probs_diff_mean:0.0030934372916817665 - training/rollout_probs_diff_std:0.00848071463406086 - training/rollout_actor_probs_pearson_corr:0.9993887543678284 - rollout_corr/rollout_is_mean:0.9999690055847168 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3896545169700403e-05 - rollout_corr/rollout_is_max:1.8591357469558716 - rollout_corr/rollout_is_min:0.3342134654521942 - rollout_corr/rollout_is_std:0.027681713923811913 - rollout_corr/rollout_is_eff_sample_size:0.999234190391299 - rollout_corr/rollout_is_seq_mean:0.9999369382858276 - rollout_corr/rollout_is_seq_std:0.0011744012590497732 - rollout_corr/rollout_is_seq_max:1.003006100654602 - rollout_corr/rollout_is_seq_min:0.9960953593254089 - rollout_corr/rollout_is_seq_max_deviation:0.0039046406745910645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.354571030009538) - rollout_corr/training_log_ppl:np.float64(0.2829173822829034) - rollout_corr/kl:np.float64(0.0005067846795761) - rollout_corr/k3_kl:np.float64(0.00047523474137278754) - rollout_corr/rollout_ppl:np.float64(1.353815943468362) - rollout_corr/rollout_log_ppl:np.float64(0.28241059606079943) - rollout_corr/log_ppl_diff:np.float64(0.0005067862221039832) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010225145379081368) - rollout_corr/log_ppl_diff_max:np.float64(0.005544401705265045) - rollout_corr/log_ppl_diff_min:np.float64(-0.003694787621498108) - rollout_corr/ppl_ratio:np.float64(1.000507768476382) - rollout_corr/chi2_token:np.float64(0.0008780690841376781) - rollout_corr/chi2_seq:np.float64(0.740443660877645) - actor/pg_loss:np.float64(-8.333136793226004e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747284138422401) - actor/ppo_kl:np.float64(2.7713305339016614e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16628609038889408) - perf/mfu/actor:np.float64(0.1349622504868255) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.00208282470703) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004228602629154921 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004228602629154921 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:843.28515625 - response_length/max:3164.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:843.28515625 - response_length_non_aborted/max:3164.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013750791549682617 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8266812413930893) - timing_s/agent_loop/generate_sequences/max:np.float64(24.079156171530485) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.841072585782968) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.079156171530485) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:292 - timing_s/agent_loop/slowest/response_length:3164 - timing_s/gen:30.762977486476302 - timing_s/reward:0.07941771484911442 - timing_s/old_log_prob:3.5987925808876753 - timing_s/adv:0.14997461810708046 - timing_s/update_actor:14.10401264578104 - timing_s/step:48.78351537883282 - timing_s/stop_profile:0.00021011196076869965 - timing_per_token_ms/gen:0.14249969884555055 - timing_per_token_ms/update_actor:0.04859682193395138 - timing_per_token_ms/adv:0.0005167529265469221 - perf/total_num_tokens:290225 - perf/time_per_step:48.78351537883282 - perf/throughput:743.6554073292776 (TaskRunner pid=2281424) Training Progress: 33%|███▎ | 33/100 [33:48<1:20:06, 71.74s/it] (TaskRunner pid=2281424) step:34 - global_seqlen/min:25666 - global_seqlen/max:73494 - global_seqlen/minmax_diff:47828 - global_seqlen/balanced_min:35473 - global_seqlen/balanced_max:36595 - global_seqlen/mean:35820.75 - actor/entropy:0.35148361325263977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45545026659965515 - training/rollout_probs_diff_mean:0.0031844459008425474 - training/rollout_probs_diff_std:0.00856610108166933 - training/rollout_actor_probs_pearson_corr:0.9994761347770691 - rollout_corr/rollout_is_mean:0.9999700784683228 - rollout_corr/rollout_is_ratio_fraction_high:9.077787581190933e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8155575162381865e-05 - rollout_corr/rollout_is_max:2.0403316020965576 - rollout_corr/rollout_is_min:0.33807873725891113 - rollout_corr/rollout_is_std:0.02805599570274353 - rollout_corr/rollout_is_eff_sample_size:0.9992133611396591 - rollout_corr/rollout_is_seq_mean:0.9998204112052917 - rollout_corr/rollout_is_seq_std:0.0015267465496435761 - rollout_corr/rollout_is_seq_max:1.003962516784668 - rollout_corr/rollout_is_seq_min:0.9919791221618652 - rollout_corr/rollout_is_seq_max_deviation:0.008020877838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.309415325988084) - rollout_corr/training_log_ppl:np.float64(0.25629871332785115) - rollout_corr/kl:np.float64(0.0007180621188593861) - rollout_corr/k3_kl:np.float64(0.000561066810490729) - rollout_corr/rollout_ppl:np.float64(1.3084843065589666) - rollout_corr/rollout_log_ppl:np.float64(0.2555806517775636) - rollout_corr/log_ppl_diff:np.float64(0.0007180615502875298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012585615331772715) - rollout_corr/log_ppl_diff_max:np.float64(0.009126394987106323) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031400546431541443) - rollout_corr/ppl_ratio:np.float64(1.0007198799867183) - rollout_corr/chi2_token:np.float64(0.0008239294402301311) - rollout_corr/chi2_seq:np.float64(1.3956339918076992) - actor/pg_loss:np.float64(6.344064604490995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036903620963357753) - actor/ppo_kl:np.float64(0.00010281039390358515) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1495772898197174) - perf/mfu/actor:np.float64(0.12881182116660295) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.90487289428711) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018949881196022034 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0018949881196022034 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:860.6171875 - response_length/max:6530.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:860.6171875 - response_length_non_aborted/max:6530.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010554678738117218 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5495978090912104) - timing_s/agent_loop/generate_sequences/max:np.float64(42.11380794644356) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.242849451620714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(42.11380794644356) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:6530 - timing_s/gen:48.750448731705546 - timing_s/reward:0.07960104756057262 - timing_s/old_log_prob:3.7480743881314993 - timing_s/adv:0.1472205240279436 - timing_s/update_actor:14.973976226523519 - timing_s/step:67.78288715332747 - timing_s/stop_profile:3.6908313632011414e-05 - timing_per_token_ms/gen:0.22127310855992496 - timing_per_token_ms/update_actor:0.05225315015222852 - timing_per_token_ms/adv:0.0005137403740427811 - perf/total_num_tokens:286566 - perf/time_per_step:67.78288715332747 - perf/throughput:528.4630310740837 (TaskRunner pid=2281424) Training Progress: 34%|███▍ | 34/100 [34:56<1:17:37, 70.56s/it] (TaskRunner pid=2281424) step:35 - global_seqlen/min:22711 - global_seqlen/max:52151 - global_seqlen/minmax_diff:29440 - global_seqlen/balanced_min:33570 - global_seqlen/balanced_max:34828 - global_seqlen/mean:33923.125 - actor/entropy:0.356091171503067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23818081617355347 - training/rollout_probs_diff_mean:0.003347822232171893 - training/rollout_probs_diff_std:0.008754725567996502 - training/rollout_actor_probs_pearson_corr:0.9993891716003418 - rollout_corr/rollout_is_mean:0.999956488609314 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.74996237346204e-06 - rollout_corr/rollout_is_max:1.6463863849639893 - rollout_corr/rollout_is_min:0.14987757802009583 - rollout_corr/rollout_is_std:0.02889123558998108 - rollout_corr/rollout_is_eff_sample_size:0.9991658736928141 - rollout_corr/rollout_is_seq_mean:0.9999445676803589 - rollout_corr/rollout_is_seq_std:0.0016311032231897116 - rollout_corr/rollout_is_seq_max:1.0061169862747192 - rollout_corr/rollout_is_seq_min:0.994307279586792 - rollout_corr/rollout_is_seq_max_deviation:0.006116986274719238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3848898322321475) - rollout_corr/training_log_ppl:np.float64(0.3058403903269209) - rollout_corr/kl:np.float64(0.0004393863629239547) - rollout_corr/k3_kl:np.float64(0.0005089315998816346) - rollout_corr/rollout_ppl:np.float64(1.3842959650792181) - rollout_corr/rollout_log_ppl:np.float64(0.3054010035120882) - rollout_corr/log_ppl_diff:np.float64(0.0004393868148326874) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012084643822163343) - rollout_corr/log_ppl_diff_max:np.float64(0.006951093673706055) - rollout_corr/log_ppl_diff_min:np.float64(-0.005402863025665283) - rollout_corr/ppl_ratio:np.float64(1.0004409453831613) - rollout_corr/chi2_token:np.float64(0.001150362892076373) - rollout_corr/chi2_seq:np.float64(1.6086886166594923) - actor/pg_loss:np.float64(2.0964303985238075e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002854640780469708) - actor/ppo_kl:np.float64(-8.599662149322285e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18192359805107117) - perf/mfu/actor:np.float64(0.12649445809002435) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.95371627807617) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00438565481454134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00438565481454134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:801.28515625 - response_length/max:5012.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:801.28515625 - response_length_non_aborted/max:5012.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.070756793022156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6557252500206232) - timing_s/agent_loop/generate_sequences/max:np.float64(33.535721046850085) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.11258227860526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.535721046850085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:5012 - timing_s/gen:40.21909198909998 - timing_s/reward:0.07026992179453373 - timing_s/old_log_prob:3.5517083313316107 - timing_s/adv:0.14980567805469036 - timing_s/update_actor:14.293088844045997 - timing_s/step:58.31905349902809 - timing_s/stop_profile:2.7982518076896667e-05 - timing_per_token_ms/gen:0.19606731368602187 - timing_per_token_ms/update_actor:0.05266720284483666 - timing_per_token_ms/adv:0.0005520042672022786 - perf/total_num_tokens:271385 - perf/time_per_step:58.31905349902809 - perf/throughput:581.6816797372294 (TaskRunner pid=2281424) Training Progress: 35%|███▌ | 35/100 [35:55<1:12:28, 66.89s/it] (TaskRunner pid=2281424) step:36 - global_seqlen/min:33717 - global_seqlen/max:49536 - global_seqlen/minmax_diff:15819 - global_seqlen/balanced_min:43612 - global_seqlen/balanced_max:43672 - global_seqlen/mean:43658.375 - actor/entropy:0.362861692905426 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8421130180358887 - training/rollout_probs_diff_mean:0.003083376446738839 - training/rollout_probs_diff_std:0.00820560660213232 - training/rollout_actor_probs_pearson_corr:0.9995745420455933 - rollout_corr/rollout_is_mean:0.9999458193778992 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.136867225199239e-06 - rollout_corr/rollout_is_max:1.5991979837417603 - rollout_corr/rollout_is_min:0.010655050165951252 - rollout_corr/rollout_is_std:0.027175385504961014 - rollout_corr/rollout_is_eff_sample_size:0.9992619243999026 - rollout_corr/rollout_is_seq_mean:0.999880313873291 - rollout_corr/rollout_is_seq_std:0.0011923309648409486 - rollout_corr/rollout_is_seq_max:1.0057610273361206 - rollout_corr/rollout_is_seq_min:0.9923911094665527 - rollout_corr/rollout_is_seq_max_deviation:0.007608890533447266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3939749002456665) - rollout_corr/training_log_ppl:np.float64(0.3143607703241287) - rollout_corr/kl:np.float64(0.0004875935232391271) - rollout_corr/k3_kl:np.float64(0.00043940715849544176) - rollout_corr/rollout_ppl:np.float64(1.3933153660036623) - rollout_corr/rollout_log_ppl:np.float64(0.3138731736544287) - rollout_corr/log_ppl_diff:np.float64(0.0004875966696999967) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008989509078674018) - rollout_corr/log_ppl_diff_max:np.float64(0.008858025074005127) - rollout_corr/log_ppl_diff_min:np.float64(-0.003800034523010254) - rollout_corr/ppl_ratio:np.float64(1.0004885424859822) - rollout_corr/chi2_token:np.float64(0.0007559196092188358) - rollout_corr/chi2_seq:np.float64(0.28486403776332736) - actor/pg_loss:np.float64(4.073767922818661e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001381784221621274) - actor/ppo_kl:np.float64(-3.878047037275678e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11497985571622849) - perf/mfu/actor:np.float64(0.15230643812180578) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.80973815917969) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006595357321202755 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006595357321202755 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1094.66796875 - response_length/max:3514.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1094.66796875 - response_length_non_aborted/max:3514.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7200207114219666e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3092289455235004) - timing_s/agent_loop/generate_sequences/max:np.float64(27.297075459733605) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.346446073104744) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.297075459733605) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:3514 - timing_s/gen:34.776961313560605 - timing_s/reward:0.08570255897939205 - timing_s/old_log_prob:3.737231995910406 - timing_s/adv:0.14758271723985672 - timing_s/update_actor:15.371759809553623 - timing_s/step:54.21316165663302 - timing_s/stop_profile:2.7824193239212036e-05 - timing_per_token_ms/gen:0.12409927851110891 - timing_per_token_ms/update_actor:0.04401148636874833 - timing_per_token_ms/adv:0.00042254984650670326 - perf/total_num_tokens:349267 - perf/time_per_step:54.21316165663302 - perf/throughput:805.3095164697587 (TaskRunner pid=2281424) Training Progress: 36%|███▌ | 36/100 [36:49<1:07:18, 63.10s/it] (TaskRunner pid=2281424) step:37 - global_seqlen/min:34704 - global_seqlen/max:44098 - global_seqlen/minmax_diff:9394 - global_seqlen/balanced_min:39539 - global_seqlen/balanced_max:39624 - global_seqlen/mean:39597.875 - actor/entropy:0.35228487849235535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413027286529541 - training/rollout_probs_diff_mean:0.003145897062495351 - training/rollout_probs_diff_std:0.0083078658208251 - training/rollout_actor_probs_pearson_corr:0.9995119571685791 - rollout_corr/rollout_is_mean:0.9999536275863647 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.998736247012857e-06 - rollout_corr/rollout_is_max:1.6415951251983643 - rollout_corr/rollout_is_min:0.39738908410072327 - rollout_corr/rollout_is_std:0.027896204963326454 - rollout_corr/rollout_is_eff_sample_size:0.9992224068794121 - rollout_corr/rollout_is_seq_mean:0.9999120235443115 - rollout_corr/rollout_is_seq_std:0.0014193912502378225 - rollout_corr/rollout_is_seq_max:1.0059196949005127 - rollout_corr/rollout_is_seq_min:0.9947477579116821 - rollout_corr/rollout_is_seq_max_deviation:0.005919694900512695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3658450348302722) - rollout_corr/training_log_ppl:np.float64(0.29802232861402445) - rollout_corr/kl:np.float64(0.0005081376050801722) - rollout_corr/k3_kl:np.float64(0.00047723877818839355) - rollout_corr/rollout_ppl:np.float64(1.365188850555569) - rollout_corr/rollout_log_ppl:np.float64(0.2975141915230779) - rollout_corr/log_ppl_diff:np.float64(0.0005081370909465477) - rollout_corr/log_ppl_abs_diff:np.float64(0.001064729891368188) - rollout_corr/log_ppl_diff_max:np.float64(0.005850955843925476) - rollout_corr/log_ppl_diff_min:np.float64(-0.00405728816986084) - rollout_corr/ppl_ratio:np.float64(1.000509355450049) - rollout_corr/chi2_token:np.float64(0.0009017251431941986) - rollout_corr/chi2_seq:np.float64(1.061360603896901) - actor/pg_loss:np.float64(6.26088585704565e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021849221991487866) - actor/ppo_kl:np.float64(-2.4668701797736503e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1671565007418394) - perf/mfu/actor:np.float64(0.14026788981271016) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.84344863891602) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011376905255019665 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011376905255019665 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:976.87109375 - response_length/max:3632.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:976.87109375 - response_length_non_aborted/max:3632.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6365742087364197e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.545783119276166) - timing_s/agent_loop/generate_sequences/max:np.float64(27.446757026016712) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.701898429557332) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.446757026016712) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:3632 - timing_s/gen:33.87001137807965 - timing_s/reward:0.0845334343612194 - timing_s/old_log_prob:3.622224699705839 - timing_s/adv:0.14574186690151691 - timing_s/update_actor:15.075419383123517 - timing_s/step:52.884285079315305 - timing_s/stop_profile:0.00012076087296009064 - timing_per_token_ms/gen:0.13543724734215848 - timing_per_token_ms/update_actor:0.0475891047913667 - timing_per_token_ms/adv:0.0004600684598021892 - perf/total_num_tokens:316783 - perf/time_per_step:52.884285079315305 - perf/throughput:748.7644947948435 (TaskRunner pid=2281424) Training Progress: 37%|███▋ | 37/100 [37:42<1:03:03, 60.05s/it] (TaskRunner pid=2281424) step:38 - global_seqlen/min:25876 - global_seqlen/max:56353 - global_seqlen/minmax_diff:30477 - global_seqlen/balanced_min:36571 - global_seqlen/balanced_max:36714 - global_seqlen/mean:36659.625 - actor/entropy:0.32807788252830505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9003579616546631 - training/rollout_probs_diff_mean:0.002977442229166627 - training/rollout_probs_diff_std:0.008599979802966118 - training/rollout_actor_probs_pearson_corr:0.9994491338729858 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:4.4448197513702326e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3334459254110698e-05 - rollout_corr/rollout_is_max:3.384216547012329 - rollout_corr/rollout_is_min:0.005166077055037022 - rollout_corr/rollout_is_std:0.027199501171708107 - rollout_corr/rollout_is_eff_sample_size:0.999260734067485 - rollout_corr/rollout_is_seq_mean:0.9999963641166687 - rollout_corr/rollout_is_seq_std:0.0012283420655876398 - rollout_corr/rollout_is_seq_max:1.00580632686615 - rollout_corr/rollout_is_seq_min:0.9936442375183105 - rollout_corr/rollout_is_seq_max_deviation:0.006355762481689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3002572190016508) - rollout_corr/training_log_ppl:np.float64(0.24828266966505907) - rollout_corr/kl:np.float64(0.0003652476746376365) - rollout_corr/k3_kl:np.float64(0.00043928875018650615) - rollout_corr/rollout_ppl:np.float64(1.2997558075003326) - rollout_corr/rollout_log_ppl:np.float64(0.2479174185282318) - rollout_corr/log_ppl_diff:np.float64(0.0003652511368272826) - rollout_corr/log_ppl_abs_diff:np.float64(0.000950878209550865) - rollout_corr/log_ppl_diff_max:np.float64(0.008495032787322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.004545837640762329) - rollout_corr/ppl_ratio:np.float64(1.0003662216477096) - rollout_corr/chi2_token:np.float64(0.0010224031284451485) - rollout_corr/chi2_seq:np.float64(0.804062022594735) - actor/pg_loss:np.float64(-4.1241059079766273e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001981328330202814) - actor/ppo_kl:np.float64(-5.4830987617826565e-05) - actor/pg_clipfrac_lower:np.float64(3.98596921513672e-06) - actor/grad_norm:np.float64(0.12819637544453144) - perf/mfu/actor:np.float64(0.13466560382170992) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.90673160552979) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004230912774801254 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004230912774801254 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:878.83203125 - response_length/max:3854.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:878.83203125 - response_length_non_aborted/max:3854.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014106743037700653 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5907990671694279) - timing_s/agent_loop/generate_sequences/max:np.float64(27.61575059592724) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.814689949009335) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.61575059592724) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:3674 - timing_s/gen:35.19211849011481 - timing_s/reward:0.0795986708253622 - timing_s/old_log_prob:3.5315878726541996 - timing_s/adv:0.1513067688792944 - timing_s/update_actor:14.461832895874977 - timing_s/step:53.504376608878374 - timing_s/stop_profile:0.00011487491428852081 - timing_per_token_ms/gen:0.15642262453324862 - timing_per_token_ms/update_actor:0.04931117304076002 - timing_per_token_ms/adv:0.0005159176099022235 - perf/total_num_tokens:293277 - perf/time_per_step:53.504376608878374 - perf/throughput:685.1705846044154 (TaskRunner pid=2281424) Training Progress: 38%|███▊ | 38/100 [38:35<1:00:02, 58.10s/it] (TaskRunner pid=2281424) step:39 - global_seqlen/min:19213 - global_seqlen/max:40355 - global_seqlen/minmax_diff:21142 - global_seqlen/balanced_min:28562 - global_seqlen/balanced_max:29340 - global_seqlen/mean:28757.375 - actor/entropy:0.3156418800354004 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22961103916168213 - training/rollout_probs_diff_mean:0.003309106919914484 - training/rollout_probs_diff_std:0.00906699150800705 - training/rollout_actor_probs_pearson_corr:0.9993242025375366 - rollout_corr/rollout_is_mean:1.0000478029251099 - rollout_corr/rollout_is_ratio_fraction_high:1.219846944877645e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.099234724388225e-06 - rollout_corr/rollout_is_max:3.2739274501800537 - rollout_corr/rollout_is_min:0.49741330742836 - rollout_corr/rollout_is_std:0.029006537050008774 - rollout_corr/rollout_is_eff_sample_size:0.9991593281581235 - rollout_corr/rollout_is_seq_mean:1.0000423192977905 - rollout_corr/rollout_is_seq_std:0.0014898530207574368 - rollout_corr/rollout_is_seq_max:1.010595679283142 - rollout_corr/rollout_is_seq_min:0.9943899512290955 - rollout_corr/rollout_is_seq_max_deviation:0.01059567928314209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3278274578042328) - rollout_corr/training_log_ppl:np.float64(0.26448297980823554) - rollout_corr/kl:np.float64(0.00044818446750127805) - rollout_corr/k3_kl:np.float64(0.00045055161507434605) - rollout_corr/rollout_ppl:np.float64(1.3272340912371874) - rollout_corr/rollout_log_ppl:np.float64(0.264034794352483) - rollout_corr/log_ppl_diff:np.float64(0.00044818545575253665) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011384486278984696) - rollout_corr/log_ppl_diff_max:np.float64(0.006218463182449341) - rollout_corr/log_ppl_diff_min:np.float64(-0.00863182544708252) - rollout_corr/ppl_ratio:np.float64(1.0004495589528233) - rollout_corr/chi2_token:np.float64(0.0009194693993777037) - rollout_corr/chi2_seq:np.float64(0.4609958524815738) - actor/pg_loss:np.float64(1.4589284546673298e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.937741061894485e-05) - actor/ppo_kl:np.float64(4.479127906620306e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10549908224493265) - perf/mfu/actor:np.float64(0.11212622068994285) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.87326049804688) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009453050792217255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009453050792217255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:640.44921875 - response_length/max:3306.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:640.44921875 - response_length_non_aborted/max:3306.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001562461256980896 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4932099115103483) - timing_s/agent_loop/generate_sequences/max:np.float64(23.016741577535868) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.735508493344241) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.016741577535868) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:3306 - timing_s/gen:30.13746011815965 - timing_s/reward:0.07099534384906292 - timing_s/old_log_prob:3.4354004375636578 - timing_s/adv:0.1418602205812931 - timing_s/update_actor:13.508899280801415 - timing_s/step:47.37992210872471 - timing_s/stop_profile:7.83577561378479e-05 - timing_per_token_ms/gen:0.18381543788331953 - timing_per_token_ms/update_actor:0.05871928192681623 - timing_per_token_ms/adv:0.0006166253899273365 - perf/total_num_tokens:230059 - perf/time_per_step:47.37992210872471 - perf/throughput:606.9527707118056 (TaskRunner pid=2281424) Training Progress: 39%|███▉ | 39/100 [39:23<55:49, 54.90s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the particle's charge, we can use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Where: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, substitute the known values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, convert coulombs to picocoulombs ($1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C}$): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) So, the closest answer is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2281424) step:40 - global_seqlen/min:25284 - global_seqlen/max:51779 - global_seqlen/minmax_diff:26495 - global_seqlen/balanced_min:36707 - global_seqlen/balanced_max:38750 - global_seqlen/mean:36970.5 - actor/entropy:0.2809111177921295 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35805636644363403 - training/rollout_probs_diff_mean:0.0029524413403123617 - training/rollout_probs_diff_std:0.008548099547624588 - training/rollout_actor_probs_pearson_corr:0.9994715452194214 - rollout_corr/rollout_is_mean:0.9999876022338867 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.368109330243897e-06 - rollout_corr/rollout_is_max:1.9496772289276123 - rollout_corr/rollout_is_min:0.451233834028244 - rollout_corr/rollout_is_std:0.026926398277282715 - rollout_corr/rollout_is_eff_sample_size:0.999275494389906 - rollout_corr/rollout_is_seq_mean:0.9999891519546509 - rollout_corr/rollout_is_seq_std:0.0013239040272310376 - rollout_corr/rollout_is_seq_max:1.0059114694595337 - rollout_corr/rollout_is_seq_min:0.9929170608520508 - rollout_corr/rollout_is_seq_max_deviation:0.007082939147949219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2970258207060397) - rollout_corr/training_log_ppl:np.float64(0.24729529570322484) - rollout_corr/kl:np.float64(0.00048714316951858905) - rollout_corr/k3_kl:np.float64(0.00042656138907659624) - rollout_corr/rollout_ppl:np.float64(1.2963653528131545) - rollout_corr/rollout_log_ppl:np.float64(0.24680815133615397) - rollout_corr/log_ppl_diff:np.float64(0.0004871443670708686) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010458430915605277) - rollout_corr/log_ppl_diff_max:np.float64(0.007907375693321228) - rollout_corr/log_ppl_diff_min:np.float64(-0.00467248260974884) - rollout_corr/ppl_ratio:np.float64(1.0004882379435003) - rollout_corr/chi2_token:np.float64(0.0007523114327341318) - rollout_corr/chi2_seq:np.float64(0.4919047087896615) - actor/pg_loss:np.float64(0.00013426528312265873) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.462851380703796e-05) - actor/ppo_kl:np.float64(9.811125503489393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10963228065520525) - perf/mfu/actor:np.float64(0.13216087132668086) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.88793182373047) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62109375) - val-aux/sciknoweval/reward/std@16:np.float64(0.20978195620494514) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7098125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19512127996631243) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5332000000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16365501611769506) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.62185) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.209297903919098) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7927750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15226235422617695) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.47158749999999994) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10047297589092027) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6512499999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1840588452797897) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8536250000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09931525131049548) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43674999999999997) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.05502894877052068) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6747375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15323121047661067) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8911) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05002471514195125) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.416225) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03088195571501212) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6931875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.12136011366811209) - val-aux/sciknoweval/score/mean@16:np.float64(0.62109375) - val-aux/sciknoweval/score/std@16:np.float64(0.20978195620494514) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7098125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19512127996631243) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5332000000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16365501611769506) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.62185) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.209297903919098) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7927750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15226235422617695) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.47158749999999994) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10047297589092027) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6512499999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1840588452797897) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8536250000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09931525131049548) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43674999999999997) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.05502894877052068) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6747375) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15323121047661067) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8911) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05002471514195125) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.416225) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03088195571501212) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6931875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.12136011366811209) - val-core/sciknoweval/acc/mean@16:np.float64(0.62109375) - val-aux/sciknoweval/acc/std@16:np.float64(0.20978195620494514) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7098125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19512127996631243) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5332000000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16365501611769506) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.62185) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.209297903919098) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7927750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15226235422617695) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.47158749999999994) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10047297589092027) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6512499999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1840588452797897) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8536250000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09931525131049548) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43674999999999997) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.05502894877052068) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6747375) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15323121047661067) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8911) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05002471514195125) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.416225) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03088195571501212) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6931875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.12136011366811209) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.93046875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.12202913367678088) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9769125000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.06539542499226024) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8835625) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.13970718792907938) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9306750000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.1211440536861494) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9951250000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.02218746491637611) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.823675) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.14131489814444675) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.956325) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.09419032337440465) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9994624999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.004248820241722828) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.760775) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.12100807936316986) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9767624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.06242768994051049) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7088500000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08327809515343695) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99055) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.030619804261104205) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065811024978756905 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065811024978756905 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:894.265625 - response_length/max:5469.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:894.265625 - response_length_non_aborted/max:5469.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.171664714813232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.883589567616582) - timing_s/agent_loop/generate_sequences/max:np.float64(36.48411184921861) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.177588642422052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.48411184921861) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:5469 - timing_s/gen:43.124824890866876 - timing_s/reward:0.08122630603611469 - timing_s/old_log_prob:3.681945353746414 - timing_s/adv:0.15046975389122963 - timing_s/update_actor:14.803198579698801 - timing_s/step:61.928098333999515 - timing_s/testing:162.62985380925238 - timing_s/save_checkpoint:12.925621895119548 - timing_s/stop_profile:0.00014221295714378357 - timing_per_token_ms/gen:0.1883739489930061 - timing_per_token_ms/update_actor:0.0500507113093507 - timing_per_token_ms/adv:0.0005087493876578272 - perf/total_num_tokens:295764 - perf/time_per_step:61.928098333999515 - perf/throughput:596.9907197958088 (TaskRunner pid=2281424) Training Progress: 40%|████ | 40/100 [43:20<1:49:42, 109.70s/it] (TaskRunner pid=2281424) step:41 - global_seqlen/min:29658 - global_seqlen/max:51620 - global_seqlen/minmax_diff:21962 - global_seqlen/balanced_min:38538 - global_seqlen/balanced_max:38782 - global_seqlen/mean:38637.75 - actor/entropy:0.33072540163993835 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35299545526504517 - training/rollout_probs_diff_mean:0.0031244975980371237 - training/rollout_probs_diff_std:0.008380025625228882 - training/rollout_actor_probs_pearson_corr:0.9994889497756958 - rollout_corr/rollout_is_mean:1.000058650970459 - rollout_corr/rollout_is_ratio_fraction_high:1.260218232346233e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.29278564453125 - rollout_corr/rollout_is_min:0.55013507604599 - rollout_corr/rollout_is_std:0.027836313471198082 - rollout_corr/rollout_is_eff_sample_size:0.999225739561659 - rollout_corr/rollout_is_seq_mean:0.9999604225158691 - rollout_corr/rollout_is_seq_std:0.0012300461530685425 - rollout_corr/rollout_is_seq_max:1.006003737449646 - rollout_corr/rollout_is_seq_min:0.9945253133773804 - rollout_corr/rollout_is_seq_max_deviation:0.006003737449645996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3499207119457424) - rollout_corr/training_log_ppl:np.float64(0.2855829408799764) - rollout_corr/kl:np.float64(0.0005269990794536383) - rollout_corr/k3_kl:np.float64(0.0004426697283577141) - rollout_corr/rollout_ppl:np.float64(1.349194892682135) - rollout_corr/rollout_log_ppl:np.float64(0.28505594066518825) - rollout_corr/log_ppl_diff:np.float64(0.0005270002147881314) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011361698125256225) - rollout_corr/log_ppl_diff_max:np.float64(0.005944006145000458) - rollout_corr/log_ppl_diff_min:np.float64(-0.005369067192077637) - rollout_corr/ppl_ratio:np.float64(1.0005282380152494) - rollout_corr/chi2_token:np.float64(0.0007350665982812643) - rollout_corr/chi2_seq:np.float64(4.942351899808273) - actor/pg_loss:np.float64(4.62154857814312e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001543072065146589) - actor/ppo_kl:np.float64(0.0001058366084052409) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12146356515586376) - perf/mfu/actor:np.float64(0.13938522742671805) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.29471588134766) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004173422697931528 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004173422697931528 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:929.8984375 - response_length/max:4125.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:929.8984375 - response_length_non_aborted/max:4125.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015160441398620605 - timing_s/agent_loop/generate_sequences/min:np.float64(1.333501910790801) - timing_s/agent_loop/generate_sequences/max:np.float64(29.424998305737972) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.613472268974874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.424998305737972) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:4125 - timing_s/gen:35.98993204906583 - timing_s/reward:0.08053253218531609 - timing_s/old_log_prob:3.700353318825364 - timing_s/adv:0.16254926659166813 - timing_s/update_actor:14.687377415597439 - timing_s/step:54.72286199964583 - timing_s/stop_profile:0.00014472007751464844 - timing_per_token_ms/gen:0.15118389965749715 - timing_per_token_ms/update_actor:0.04751628076038796 - timing_per_token_ms/adv:0.0005258758163702213 - perf/total_num_tokens:309102 - perf/time_per_step:54.72286199964583 - perf/throughput:706.0623035441762 (TaskRunner pid=2281424) Training Progress: 41%|████ | 41/100 [44:15<1:31:40, 93.23s/it] (TaskRunner pid=2281424) step:42 - global_seqlen/min:34807 - global_seqlen/max:57617 - global_seqlen/minmax_diff:22810 - global_seqlen/balanced_min:46153 - global_seqlen/balanced_max:46569 - global_seqlen/mean:46354.0 - actor/entropy:0.4125242531299591 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730005979537964 - training/rollout_probs_diff_mean:0.0032492319587618113 - training/rollout_probs_diff_std:0.00808805413544178 - training/rollout_actor_probs_pearson_corr:0.9996025562286377 - rollout_corr/rollout_is_mean:0.9999741911888123 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.913554094964638e-06 - rollout_corr/rollout_is_max:1.8443630933761597 - rollout_corr/rollout_is_min:0.3683568835258484 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992026492912346 - rollout_corr/rollout_is_seq_mean:1.0000536441802979 - rollout_corr/rollout_is_seq_std:0.0012193896109238267 - rollout_corr/rollout_is_seq_max:1.009535312652588 - rollout_corr/rollout_is_seq_min:0.9968177080154419 - rollout_corr/rollout_is_seq_max_deviation:0.00953531265258789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4070447161793709) - rollout_corr/training_log_ppl:np.float64(0.3194918569060974) - rollout_corr/kl:np.float64(0.00035904505688222343) - rollout_corr/k3_kl:np.float64(0.00043011571953854855) - rollout_corr/rollout_ppl:np.float64(1.4064954705536366) - rollout_corr/rollout_log_ppl:np.float64(0.3191328103421256) - rollout_corr/log_ppl_diff:np.float64(0.0003590465639717877) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009108688100241125) - rollout_corr/log_ppl_diff_max:np.float64(0.0039010122418403625) - rollout_corr/log_ppl_diff_min:np.float64(-0.003787517547607422) - rollout_corr/ppl_ratio:np.float64(1.0003597426693887) - rollout_corr/chi2_token:np.float64(0.0010020083282142878) - rollout_corr/chi2_seq:np.float64(0.939779143081978) - actor/pg_loss:np.float64(0.00010238576214760542) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001587477142379612) - actor/ppo_kl:np.float64(2.258576988367622e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11312472820281982) - perf/mfu/actor:np.float64(0.15362174541859897) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.15521240234375) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01894480176270008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01894480176270008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1182.09375 - response_length/max:6248.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1182.09375 - response_length_non_aborted/max:6248.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.07406210899353e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8849900215864182) - timing_s/agent_loop/generate_sequences/max:np.float64(42.688118137419224) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.67329497004539) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(42.688118137419224) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:6248 - timing_s/gen:50.097527500241995 - timing_s/reward:0.09398127906024456 - timing_s/old_log_prob:3.954119984060526 - timing_s/adv:0.14543932490050793 - timing_s/update_actor:16.064200446009636 - timing_s/step:70.44157047756016 - timing_s/stop_profile:0.00011380389332771301 - timing_per_token_ms/gen:0.16554817822006104 - timing_per_token_ms/update_actor:0.04331934796891756 - timing_per_token_ms/adv:0.00039219734246372464 - perf/total_num_tokens:370832 - perf/time_per_step:70.44157047756016 - perf/throughput:658.0489288603596 (TaskRunner pid=2281424) Training Progress: 42%|████▏ | 42/100 [45:26<1:23:31, 86.41s/it] (TaskRunner pid=2281424) step:43 - global_seqlen/min:27945 - global_seqlen/max:57053 - global_seqlen/minmax_diff:29108 - global_seqlen/balanced_min:46139 - global_seqlen/balanced_max:46442 - global_seqlen/mean:46394.125 - actor/entropy:0.3777778148651123 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24276472628116608 - training/rollout_probs_diff_mean:0.0031969803385436535 - training/rollout_probs_diff_std:0.008178768679499626 - training/rollout_actor_probs_pearson_corr:0.9996250867843628 - rollout_corr/rollout_is_mean:0.9999624490737915 - rollout_corr/rollout_is_ratio_fraction_high:3.310370402687113e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.620740805374226e-06 - rollout_corr/rollout_is_max:3.3981943130493164 - rollout_corr/rollout_is_min:0.40390288829803467 - rollout_corr/rollout_is_std:0.027761269360780716 - rollout_corr/rollout_is_eff_sample_size:0.9992297864199887 - rollout_corr/rollout_is_seq_mean:0.9999164938926697 - rollout_corr/rollout_is_seq_std:0.0013294965028762817 - rollout_corr/rollout_is_seq_max:1.0057275295257568 - rollout_corr/rollout_is_seq_min:0.9940543174743652 - rollout_corr/rollout_is_seq_max_deviation:0.005945682525634766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.368367766495794) - rollout_corr/training_log_ppl:np.float64(0.297560689446982) - rollout_corr/kl:np.float64(0.0004662561269233034) - rollout_corr/k3_kl:np.float64(0.00041893447971119713) - rollout_corr/rollout_ppl:np.float64(1.3677249136380851) - rollout_corr/rollout_log_ppl:np.float64(0.29709443140018266) - rollout_corr/log_ppl_diff:np.float64(0.00046625804679933935) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009343025594716892) - rollout_corr/log_ppl_diff_max:np.float64(0.006767094135284424) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033786147832870483) - rollout_corr/ppl_ratio:np.float64(1.000467125326395) - rollout_corr/chi2_token:np.float64(0.0007502012886106968) - rollout_corr/chi2_seq:np.float64(0.8961137712467462) - actor/pg_loss:np.float64(7.715250831097364e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018895000766860903) - actor/ppo_kl:np.float64(-1.2429667751945317e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12724709697067738) - perf/mfu/actor:np.float64(0.15634425835350255) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.0877799987793) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015876950696110725 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015876950696110725 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:1180.00390625 - response_length/max:5656.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1180.00390625 - response_length_non_aborted/max:5656.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011191703379154205 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1672064159065485) - timing_s/agent_loop/generate_sequences/max:np.float64(40.370598992332816) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.840922129631508) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(40.370598992332816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:5656 - timing_s/gen:47.32844007387757 - timing_s/reward:0.09296286478638649 - timing_s/old_log_prob:3.898196619004011 - timing_s/adv:0.13054160960018635 - timing_s/update_actor:15.71086062118411 - timing_s/step:67.24753057211637 - timing_s/stop_profile:2.9379501938819885e-05 - timing_per_token_ms/gen:0.15667466697302238 - timing_per_token_ms/update_actor:0.04232987641534384 - timing_per_token_ms/adv:0.00035171912823063896 - perf/total_num_tokens:371153 - perf/time_per_step:67.24753057211637 - perf/throughput:689.9007979221907 (TaskRunner pid=2281424) Training Progress: 43%|████▎ | 43/100 [46:33<1:16:37, 80.66s/it] (TaskRunner pid=2281424) step:44 - global_seqlen/min:31039 - global_seqlen/max:81483 - global_seqlen/minmax_diff:50444 - global_seqlen/balanced_min:49966 - global_seqlen/balanced_max:50221 - global_seqlen/mean:50039.375 - actor/entropy:0.42505836486816406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40381795167922974 - training/rollout_probs_diff_mean:0.0033156690187752247 - training/rollout_probs_diff_std:0.00811709649860859 - training/rollout_actor_probs_pearson_corr:0.9995504021644592 - rollout_corr/rollout_is_mean:0.9999823570251465 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.108491212828085e-06 - rollout_corr/rollout_is_max:1.868242859840393 - rollout_corr/rollout_is_min:0.16989949345588684 - rollout_corr/rollout_is_std:0.028271863237023354 - rollout_corr/rollout_is_eff_sample_size:0.9992012210621306 - rollout_corr/rollout_is_seq_mean:1.0000169277191162 - rollout_corr/rollout_is_seq_std:0.0010278340196236968 - rollout_corr/rollout_is_seq_max:1.0032761096954346 - rollout_corr/rollout_is_seq_min:0.9966418147087097 - rollout_corr/rollout_is_seq_max_deviation:0.003358185291290283 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4229408679530025) - rollout_corr/training_log_ppl:np.float64(0.3282105611579027) - rollout_corr/kl:np.float64(0.0004335679858704644) - rollout_corr/k3_kl:np.float64(0.00045822281549590116) - rollout_corr/rollout_ppl:np.float64(1.422240549698472) - rollout_corr/rollout_log_ppl:np.float64(0.3277769897831604) - rollout_corr/log_ppl_diff:np.float64(0.0004335713747423142) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009576250740792602) - rollout_corr/log_ppl_diff_max:np.float64(0.004134416580200195) - rollout_corr/log_ppl_diff_min:np.float64(-0.004089929163455963) - rollout_corr/ppl_ratio:np.float64(1.000434412388131) - rollout_corr/chi2_token:np.float64(0.0009704537224024534) - rollout_corr/chi2_seq:np.float64(1.733142590848729) - actor/pg_loss:np.float64(2.68502626568079e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020757744624688712) - actor/ppo_kl:np.float64(6.416326252889348e-05) - actor/pg_clipfrac_lower:np.float64(1.8610052165968227e-06) - actor/grad_norm:np.float64(0.14784160628914833) - perf/mfu/actor:np.float64(0.1614180488812359) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.14054107666016) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005857898388057947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005857898388057947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1286.57421875 - response_length/max:7615.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1286.57421875 - response_length_non_aborted/max:7615.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9996037483215332e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0687543358653784) - timing_s/agent_loop/generate_sequences/max:np.float64(51.31303711421788) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.701673694929923) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.31303711421788) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:7615 - timing_s/gen:58.36772031150758 - timing_s/reward:0.09810716658830643 - timing_s/old_log_prob:4.152743987739086 - timing_s/adv:0.15301652811467648 - timing_s/update_actor:16.473277961835265 - timing_s/step:79.3296963069588 - timing_s/stop_profile:3.1422823667526245e-05 - timing_per_token_ms/gen:0.17721395636883192 - timing_per_token_ms/update_actor:0.04115078865851958 - timing_per_token_ms/adv:0.0003822403060456802 - perf/total_num_tokens:400315 - perf/time_per_step:79.3296963069588 - perf/throughput:630.777342275172 (TaskRunner pid=2281424) Training Progress: 44%|████▍ | 44/100 [47:52<1:14:55, 80.27s/it] (TaskRunner pid=2281424) step:45 - global_seqlen/min:29259 - global_seqlen/max:67289 - global_seqlen/minmax_diff:38030 - global_seqlen/balanced_min:45428 - global_seqlen/balanced_max:45525 - global_seqlen/mean:45499.625 - actor/entropy:0.39985886216163635 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25675493478775024 - training/rollout_probs_diff_mean:0.0032300371676683426 - training/rollout_probs_diff_std:0.00817596260458231 - training/rollout_actor_probs_pearson_corr:0.9996156692504883 - rollout_corr/rollout_is_mean:0.9999929666519165 - rollout_corr/rollout_is_ratio_fraction_high:3.4166423574788496e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.4166423574788496e-06 - rollout_corr/rollout_is_max:2.063965082168579 - rollout_corr/rollout_is_min:0.2600027620792389 - rollout_corr/rollout_is_std:0.028221219778060913 - rollout_corr/rollout_is_eff_sample_size:0.9992040775244214 - rollout_corr/rollout_is_seq_mean:0.999981701374054 - rollout_corr/rollout_is_seq_std:0.0011321797501295805 - rollout_corr/rollout_is_seq_max:1.00321364402771 - rollout_corr/rollout_is_seq_min:0.9935727119445801 - rollout_corr/rollout_is_seq_max_deviation:0.006427288055419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4365097098052502) - rollout_corr/training_log_ppl:np.float64(0.33815656184742693) - rollout_corr/kl:np.float64(0.0006105463397099165) - rollout_corr/k3_kl:np.float64(0.0004829082062087764) - rollout_corr/rollout_ppl:np.float64(1.4356286954134703) - rollout_corr/rollout_log_ppl:np.float64(0.3375460135139292) - rollout_corr/log_ppl_diff:np.float64(0.000610548333497718) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009788531460799277) - rollout_corr/log_ppl_diff_max:np.float64(0.007325723767280579) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033019185066223145) - rollout_corr/ppl_ratio:np.float64(1.0006115036085248) - rollout_corr/chi2_token:np.float64(0.0007435702718794346) - rollout_corr/chi2_seq:np.float64(2.2746842054184526) - actor/pg_loss:np.float64(7.623340934514999e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001888360293378355) - actor/ppo_kl:np.float64(0.0001864508021274247) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12979234382510185) - perf/mfu/actor:np.float64(0.15628703385179005) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.251708984375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008445513434708118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008445513434708118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1143.30078125 - response_length/max:3842.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1143.30078125 - response_length_non_aborted/max:3842.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002876799553632736 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4185115825384855) - timing_s/agent_loop/generate_sequences/max:np.float64(29.632253983989358) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.899439175627776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.632253983989358) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:3842 - timing_s/gen:36.76520950347185 - timing_s/reward:0.08381190523505211 - timing_s/old_log_prob:3.9696482811123133 - timing_s/adv:0.12938232719898224 - timing_s/update_actor:15.343466022983193 - timing_s/step:56.40911629982293 - timing_s/stop_profile:0.00011620856821537018 - timing_per_token_ms/gen:0.12561357604069853 - timing_per_token_ms/update_actor:0.0421527265966016 - timing_per_token_ms/adv:0.0003554488833671218 - perf/total_num_tokens:363997 - perf/time_per_step:56.40911629982293 - perf/throughput:806.600563606823 (TaskRunner pid=2281424) Training Progress: 45%|████▌ | 45/100 [48:49<1:07:07, 73.23s/it] (TaskRunner pid=2281424) step:46 - global_seqlen/min:20945 - global_seqlen/max:94884 - global_seqlen/minmax_diff:73939 - global_seqlen/balanced_min:43085 - global_seqlen/balanced_max:44324 - global_seqlen/mean:43425.25 - actor/entropy:0.3762817084789276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5543509721755981 - training/rollout_probs_diff_mean:0.0032524620182812214 - training/rollout_probs_diff_std:0.008445695042610168 - training/rollout_actor_probs_pearson_corr:0.9995606541633606 - rollout_corr/rollout_is_mean:0.999961793422699 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4132578144199215e-05 - rollout_corr/rollout_is_max:1.6447221040725708 - rollout_corr/rollout_is_min:0.002203931100666523 - rollout_corr/rollout_is_std:0.027791311964392662 - rollout_corr/rollout_is_eff_sample_size:0.999228120062589 - rollout_corr/rollout_is_seq_mean:0.9999802112579346 - rollout_corr/rollout_is_seq_std:0.0012473923852667212 - rollout_corr/rollout_is_seq_max:1.0055023431777954 - rollout_corr/rollout_is_seq_min:0.9959372878074646 - rollout_corr/rollout_is_seq_max_deviation:0.00550234317779541 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3645080188289285) - rollout_corr/training_log_ppl:np.float64(0.28979677851020824) - rollout_corr/kl:np.float64(0.0005104832347480048) - rollout_corr/k3_kl:np.float64(0.00045839784411327855) - rollout_corr/rollout_ppl:np.float64(1.3638080032542348) - rollout_corr/rollout_log_ppl:np.float64(0.2892862949083792) - rollout_corr/log_ppl_diff:np.float64(0.0005104836018290371) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010666771559044719) - rollout_corr/log_ppl_diff_max:np.float64(0.005919963121414185) - rollout_corr/log_ppl_diff_min:np.float64(-0.004801034927368164) - rollout_corr/ppl_ratio:np.float64(1.0005116499960423) - rollout_corr/chi2_token:np.float64(0.0008005544077605009) - rollout_corr/chi2_seq:np.float64(1.0131068772170693) - actor/pg_loss:np.float64(-1.758092548698187e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.805006137890814e-05) - actor/ppo_kl:np.float64(7.538383222893685e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10063287615776062) - perf/mfu/actor:np.float64(0.14549136899234336) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.28108978271484) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.027433983981609344 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.027433983981609344 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1105.6015625 - response_length/max:8192.0 - response_length/min:136.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1105.6015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001056082546710968 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1990190539509058) - timing_s/agent_loop/generate_sequences/max:np.float64(59.229342663660645) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.871870818904426) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.229342663660645) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.95301722921431 - timing_s/reward:0.08986181393265724 - timing_s/old_log_prob:3.924274370074272 - timing_s/adv:0.1476494651287794 - timing_s/update_actor:16.007332164794207 - timing_s/step:86.20792456716299 - timing_s/stop_profile:5.9446319937705994e-05 - timing_per_token_ms/gen:0.23302153532513517 - timing_per_token_ms/update_actor:0.04607725967263921 - timing_per_token_ms/adv:0.0004250104061829794 - perf/total_num_tokens:347402 - perf/time_per_step:86.20792456716299 - perf/throughput:503.72689306733275 (TaskRunner pid=2281424) Training Progress: 46%|████▌ | 46/100 [50:15<1:09:24, 77.13s/it] (TaskRunner pid=2281424) step:47 - global_seqlen/min:31035 - global_seqlen/max:67357 - global_seqlen/minmax_diff:36322 - global_seqlen/balanced_min:49399 - global_seqlen/balanced_max:49649 - global_seqlen/mean:49578.625 - actor/entropy:0.34646978974342346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5756693482398987 - training/rollout_probs_diff_mean:0.0030399137176573277 - training/rollout_probs_diff_std:0.00816922727972269 - training/rollout_actor_probs_pearson_corr:0.9995539784431458 - rollout_corr/rollout_is_mean:0.9999984502792358 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.122617484536022e-06 - rollout_corr/rollout_is_max:1.9699045419692993 - rollout_corr/rollout_is_min:0.25761306285858154 - rollout_corr/rollout_is_std:0.026937464252114296 - rollout_corr/rollout_is_eff_sample_size:0.9992746611342851 - rollout_corr/rollout_is_seq_mean:1.0000650882720947 - rollout_corr/rollout_is_seq_std:0.001198867685161531 - rollout_corr/rollout_is_seq_max:1.005571961402893 - rollout_corr/rollout_is_seq_min:0.9964936971664429 - rollout_corr/rollout_is_seq_max_deviation:0.005571961402893066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3529047267511487) - rollout_corr/training_log_ppl:np.float64(0.28667624859372154) - rollout_corr/kl:np.float64(0.0004597161929975613) - rollout_corr/k3_kl:np.float64(0.00046043861678413123) - rollout_corr/rollout_ppl:np.float64(1.3522660555317998) - rollout_corr/rollout_log_ppl:np.float64(0.2862165296683088) - rollout_corr/log_ppl_diff:np.float64(0.0004597189254127443) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009910539374686778) - rollout_corr/log_ppl_diff_max:np.float64(0.007551088929176331) - rollout_corr/log_ppl_diff_min:np.float64(-0.004575580358505249) - rollout_corr/ppl_ratio:np.float64(1.0004606684669852) - rollout_corr/chi2_token:np.float64(0.0009119359310716391) - rollout_corr/chi2_seq:np.float64(954.7251316532493) - actor/pg_loss:np.float64(-8.133123628795147e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001264397735667444) - actor/ppo_kl:np.float64(0.0001422462675852998) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11066612228751183) - perf/mfu/actor:np.float64(0.1590812815304962) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.31835174560547) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024808576330542564 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024808576330542564 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1284.58203125 - response_length/max:7270.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1284.58203125 - response_length_non_aborted/max:7270.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.761533975601196e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.059980768710375) - timing_s/agent_loop/generate_sequences/max:np.float64(50.74220941402018) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.601909340250131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.74220941402018) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:7270 - timing_s/gen:58.42462229728699 - timing_s/reward:0.09585716761648655 - timing_s/old_log_prob:3.987767066806555 - timing_s/adv:0.1361205969005823 - timing_s/update_actor:16.680994622409344 - timing_s/step:79.40998443961143 - timing_s/stop_profile:0.00011073052883148193 - timing_per_token_ms/gen:0.17766181940650377 - timing_per_token_ms/update_actor:0.04205692125994152 - timing_per_token_ms/adv:0.0003431937576440006 - perf/total_num_tokens:396629 - perf/time_per_step:79.40998443961143 - perf/throughput:624.3374224270607 (TaskRunner pid=2281424) Training Progress: 47%|████▋ | 47/100 [51:35<1:08:44, 77.83s/it] (TaskRunner pid=2281424) step:48 - global_seqlen/min:22232 - global_seqlen/max:57431 - global_seqlen/minmax_diff:35199 - global_seqlen/balanced_min:41913 - global_seqlen/balanced_max:42517 - global_seqlen/mean:42023.25 - actor/entropy:0.34810274839401245 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931787967681885 - training/rollout_probs_diff_mean:0.003345586359500885 - training/rollout_probs_diff_std:0.008651177398860455 - training/rollout_actor_probs_pearson_corr:0.9994678497314453 - rollout_corr/rollout_is_mean:1.0000156164169312 - rollout_corr/rollout_is_ratio_fraction_high:7.471328899555374e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.471328899555374e-06 - rollout_corr/rollout_is_max:2.302506685256958 - rollout_corr/rollout_is_min:0.015659699216485023 - rollout_corr/rollout_is_std:0.02865091897547245 - rollout_corr/rollout_is_eff_sample_size:0.9991797981155129 - rollout_corr/rollout_is_seq_mean:1.0000426769256592 - rollout_corr/rollout_is_seq_std:0.0014892894541844726 - rollout_corr/rollout_is_seq_max:1.0078259706497192 - rollout_corr/rollout_is_seq_min:0.9916344881057739 - rollout_corr/rollout_is_seq_max_deviation:0.008365511894226074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.361631895415485) - rollout_corr/training_log_ppl:np.float64(0.296743152604904) - rollout_corr/kl:np.float64(0.00044437152225995646) - rollout_corr/k3_kl:np.float64(0.0004750857856947732) - rollout_corr/rollout_ppl:np.float64(1.3610092177987099) - rollout_corr/rollout_log_ppl:np.float64(0.2962987821374554) - rollout_corr/log_ppl_diff:np.float64(0.0004443704674486071) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010612811020109802) - rollout_corr/log_ppl_diff_max:np.float64(0.009809985756874084) - rollout_corr/log_ppl_diff_min:np.float64(-0.005429580807685852) - rollout_corr/ppl_ratio:np.float64(1.0004456255119294) - rollout_corr/chi2_token:np.float64(0.0010020604822784662) - rollout_corr/chi2_seq:np.float64(0.4791911744978279) - actor/pg_loss:np.float64(3.651808947324753e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019356490452082653) - actor/ppo_kl:np.float64(3.127409698944206e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2193583231419325) - perf/mfu/actor:np.float64(0.14563750602154726) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.25163650512695) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024314971640706062 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024314971640706062 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1045.6640625 - response_length/max:5423.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1045.6640625 - response_length_non_aborted/max:5423.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001386590301990509 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7352543752640486) - timing_s/agent_loop/generate_sequences/max:np.float64(38.22215409949422) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.838098710140912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.22215409949422) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:5423 - timing_s/gen:45.5628403685987 - timing_s/reward:0.08806743286550045 - timing_s/old_log_prob:3.732063392177224 - timing_s/adv:0.15191545337438583 - timing_s/update_actor:15.251708237454295 - timing_s/step:64.87329536117613 - timing_s/stop_profile:0.00012004934251308441 - timing_per_token_ms/gen:0.17020748017706563 - timing_per_token_ms/update_actor:0.04536687499614587 - timing_per_token_ms/adv:0.00045187917811683365 - perf/total_num_tokens:336186 - perf/time_per_step:64.87329536117613 - perf/throughput:647.7742461831082 (TaskRunner pid=2281424) Training Progress: 48%|████▊ | 48/100 [52:40<1:04:05, 73.96s/it] (TaskRunner pid=2281424) step:49 - global_seqlen/min:23092 - global_seqlen/max:51168 - global_seqlen/minmax_diff:28076 - global_seqlen/balanced_min:37111 - global_seqlen/balanced_max:37334 - global_seqlen/mean:37259.625 - actor/entropy:0.3179267346858978 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20985785126686096 - training/rollout_probs_diff_mean:0.0031237066723406315 - training/rollout_probs_diff_std:0.008550681173801422 - training/rollout_actor_probs_pearson_corr:0.9993894100189209 - rollout_corr/rollout_is_mean:0.9999985098838806 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.58763451105915e-06 - rollout_corr/rollout_is_max:1.7586421966552734 - rollout_corr/rollout_is_min:0.3407052159309387 - rollout_corr/rollout_is_std:0.02756519615650177 - rollout_corr/rollout_is_eff_sample_size:0.9992407369068808 - rollout_corr/rollout_is_seq_mean:0.9999875426292419 - rollout_corr/rollout_is_seq_std:0.0011641301680356264 - rollout_corr/rollout_is_seq_max:1.0047152042388916 - rollout_corr/rollout_is_seq_min:0.9961039423942566 - rollout_corr/rollout_is_seq_max_deviation:0.0047152042388916016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.328462516888976) - rollout_corr/training_log_ppl:np.float64(0.2660138398205163) - rollout_corr/kl:np.float64(0.00038798783518378954) - rollout_corr/k3_kl:np.float64(0.0004385801271951095) - rollout_corr/rollout_ppl:np.float64(1.3279090919531882) - rollout_corr/rollout_log_ppl:np.float64(0.26562585204374045) - rollout_corr/log_ppl_diff:np.float64(0.0003879877767758444) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009870426292764023) - rollout_corr/log_ppl_diff_max:np.float64(0.006948456168174744) - rollout_corr/log_ppl_diff_min:np.float64(-0.004289939999580383) - rollout_corr/ppl_ratio:np.float64(1.0003889380022883) - rollout_corr/chi2_token:np.float64(0.0009887455962598324) - rollout_corr/chi2_seq:np.float64(1.5676374214235693) - actor/pg_loss:np.float64(2.256419975310564e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012307778933973168) - actor/ppo_kl:np.float64(-3.0272832692190832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07936635101214051) - perf/mfu/actor:np.float64(0.13578834962618122) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.31588363647461) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006559879519045353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006559879519045353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:909.73828125 - response_length/max:4227.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:909.73828125 - response_length_non_aborted/max:4227.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.641796350479126e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1448177974671125) - timing_s/agent_loop/generate_sequences/max:np.float64(29.94586299546063) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.983509431694984) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.94586299546063) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:4227 - timing_s/gen:37.325577680021524 - timing_s/reward:0.08239317126572132 - timing_s/old_log_prob:3.49440111592412 - timing_s/adv:0.12574291415512562 - timing_s/update_actor:14.33308171108365 - timing_s/step:55.44903611578047 - timing_s/stop_profile:0.00013232603669166565 - timing_per_token_ms/gen:0.16026921238517913 - timing_per_token_ms/update_actor:0.04808516494423807 - timing_per_token_ms/adv:0.00042184708701149576 - perf/total_num_tokens:298077 - perf/time_per_step:55.44903611578047 - perf/throughput:671.9616355855125 (TaskRunner pid=2281424) Training Progress: 49%|████▉ | 49/100 [53:35<58:09, 68.42s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the charge $ q $, we use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substituting the given values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) So the closest option is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2281424) step:50 - global_seqlen/min:34234 - global_seqlen/max:75152 - global_seqlen/minmax_diff:40918 - global_seqlen/balanced_min:51555 - global_seqlen/balanced_max:51907 - global_seqlen/mean:51805.25 - actor/entropy:0.4125359058380127 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19821900129318237 - training/rollout_probs_diff_mean:0.0032416246831417084 - training/rollout_probs_diff_std:0.008041433990001678 - training/rollout_actor_probs_pearson_corr:0.999610185623169 - rollout_corr/rollout_is_mean:1.0000464916229248 - rollout_corr/rollout_is_ratio_fraction_high:8.665861969348043e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.167043685913086 - rollout_corr/rollout_is_min:0.5511428117752075 - rollout_corr/rollout_is_std:0.028244443237781525 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:0.9999783635139465 - rollout_corr/rollout_is_seq_std:0.0010451057460159063 - rollout_corr/rollout_is_seq_max:1.0044012069702148 - rollout_corr/rollout_is_seq_min:0.9962003827095032 - rollout_corr/rollout_is_seq_max_deviation:0.004401206970214844 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4209796008653939) - rollout_corr/training_log_ppl:np.float64(0.3332373514131177) - rollout_corr/kl:np.float64(0.0004479175979907879) - rollout_corr/k3_kl:np.float64(0.0004798759525215246) - rollout_corr/rollout_ppl:np.float64(1.4203380569815636) - rollout_corr/rollout_log_ppl:np.float64(0.3327894336543977) - rollout_corr/log_ppl_diff:np.float64(0.0004479177587199956) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009449601930100471) - rollout_corr/log_ppl_diff_max:np.float64(0.008978396654129028) - rollout_corr/log_ppl_diff_min:np.float64(-0.004376143217086792) - rollout_corr/ppl_ratio:np.float64(1.0004488993436098) - rollout_corr/chi2_token:np.float64(0.0010542639065533876) - rollout_corr/chi2_seq:np.float64(2.937424761708826) - actor/pg_loss:np.float64(0.00014853477478027344) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021279246669791974) - actor/ppo_kl:np.float64(1.2714361647070405e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1673903912305832) - perf/mfu/actor:np.float64(0.16524905376259622) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.29336929321289) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.65390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.21345823452859586) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7464375000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19899169203769165) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.55885) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1654343787430585) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6524) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21312019792510917) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8329625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14911841799011322) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49555000000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.0995466061193929) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6846875000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.19169144926745516) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8903625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08902310096279493) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4632625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04804278474577391) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7100125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16088561753057692) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9214874999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04076217776661745) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.448325) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.02214616706327121) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7325125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.12534302205736753) - val-aux/sciknoweval/score/mean@16:np.float64(0.65390625) - val-aux/sciknoweval/score/std@16:np.float64(0.21345823452859586) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7464375000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19899169203769165) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.55885) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1654343787430585) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6524) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21312019792510917) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8329625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14911841799011322) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49555000000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.0995466061193929) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6846875000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.19169144926745516) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8903625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08902310096279493) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4632625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.04804278474577391) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7100125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16088561753057692) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9214874999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04076217776661745) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.448325) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.02214616706327121) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7325125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.12534302205736753) - val-core/sciknoweval/acc/mean@16:np.float64(0.65390625) - val-aux/sciknoweval/acc/std@16:np.float64(0.21345823452859586) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7464375000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19899169203769165) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.55885) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1654343787430585) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6524) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21312019792510917) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8329625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14911841799011322) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49555000000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.0995466061193929) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6846875000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.19169144926745516) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8903625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08902310096279493) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4632625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.04804278474577391) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7100125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16088561753057692) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9214874999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04076217776661745) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.448325) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.02214616706327121) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7325125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.12534302205736753) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.93359375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.11242025595458031) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9786250000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.06145532564716834) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8873125) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.12815918862605175) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9337624999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11286336680590714) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9963750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.019611055703303716) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8331125) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12538427780463093) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9568625) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.08973386165612807) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0029360164781816363) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7791125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.10222723529017974) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9782) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.06007209345275405) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7367125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06895219571062065) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9924125) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03078564025657012) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012263277545571327 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012263277545571327 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1352.2890625 - response_length/max:6204.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1352.2890625 - response_length_non_aborted/max:6204.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010510161519050598 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9486863240599632) - timing_s/agent_loop/generate_sequences/max:np.float64(45.78752861917019) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.385333957761759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(45.78752861917019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:6204 - timing_s/gen:52.41064179688692 - timing_s/reward:0.09868243709206581 - timing_s/old_log_prob:4.130571527406573 - timing_s/adv:0.1609333734959364 - timing_s/update_actor:16.788390869274735 - timing_s/step:73.67946309968829 - timing_s/testing:173.9143652804196 - timing_s/save_checkpoint:12.895720075815916 - timing_s/stop_profile:0.00021344982087612152 - timing_per_token_ms/gen:0.15139445788358546 - timing_per_token_ms/update_actor:0.040508420645771266 - timing_per_token_ms/adv:0.00038831337918438863 - perf/total_num_tokens:414442 - perf/time_per_step:73.67946309968829 - perf/throughput:703.1165513503744 (TaskRunner pid=2281424) Training Progress: 50%|█████ | 50/100 [57:56<1:45:03, 126.06s/it] (TaskRunner pid=2281424) step:51 - global_seqlen/min:25192 - global_seqlen/max:76157 - global_seqlen/minmax_diff:50965 - global_seqlen/balanced_min:48271 - global_seqlen/balanced_max:48448 - global_seqlen/mean:48379.375 - actor/entropy:0.41202202439308167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37214818596839905 - training/rollout_probs_diff_mean:0.003262621583417058 - training/rollout_probs_diff_std:0.008078004233539104 - training/rollout_actor_probs_pearson_corr:0.9995943903923035 - rollout_corr/rollout_is_mean:0.9999638795852661 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.276774456637213e-06 - rollout_corr/rollout_is_max:1.636783480644226 - rollout_corr/rollout_is_min:0.13366740942001343 - rollout_corr/rollout_is_std:0.027932504191994667 - rollout_corr/rollout_is_eff_sample_size:0.999220264452564 - rollout_corr/rollout_is_seq_mean:0.9999167919158936 - rollout_corr/rollout_is_seq_std:0.001000249176286161 - rollout_corr/rollout_is_seq_max:1.0028444528579712 - rollout_corr/rollout_is_seq_min:0.9964448809623718 - rollout_corr/rollout_is_seq_max_deviation:0.003555119037628174 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3898025089874864) - rollout_corr/training_log_ppl:np.float64(0.31166147111798637) - rollout_corr/kl:np.float64(0.0004372116775031021) - rollout_corr/k3_kl:np.float64(0.0003934571469699222) - rollout_corr/rollout_ppl:np.float64(1.3891857019625604) - rollout_corr/rollout_log_ppl:np.float64(0.3112242584757041) - rollout_corr/log_ppl_diff:np.float64(0.00043721264228224754) - rollout_corr/log_ppl_abs_diff:np.float64(0.000884955981746316) - rollout_corr/log_ppl_diff_max:np.float64(0.007482245564460754) - rollout_corr/log_ppl_diff_min:np.float64(-0.002665892243385315) - rollout_corr/ppl_ratio:np.float64(1.0004379730671644) - rollout_corr/chi2_token:np.float64(0.0006916895508766174) - rollout_corr/chi2_seq:np.float64(0.8691392296459526) - actor/pg_loss:np.float64(5.567236803472042e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015443053536046136) - actor/ppo_kl:np.float64(-3.0327652694950302e-05) - actor/pg_clipfrac_lower:np.float64(8.184056241589133e-07) - actor/grad_norm:np.float64(0.10533490870147943) - perf/mfu/actor:np.float64(0.1597418217861688) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.85708999633789) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014979914762079716 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014979914762079716 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1244.66796875 - response_length/max:5675.0 - response_length/min:200.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1244.66796875 - response_length_non_aborted/max:5675.0 - response_length_non_aborted/min:200.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011126697063446045 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4992719031870365) - timing_s/agent_loop/generate_sequences/max:np.float64(40.4043846745044) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.760603568858642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(40.4043846745044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:5675 - timing_s/gen:46.7182105332613 - timing_s/reward:0.0914076529443264 - timing_s/old_log_prob:4.139765975996852 - timing_s/adv:0.15743766352534294 - timing_s/update_actor:16.27989299967885 - timing_s/step:67.497363332659 - timing_s/stop_profile:0.00012996233999729156 - timing_per_token_ms/gen:0.1466198331421887 - timing_per_token_ms/update_actor:0.04206310281932862 - timing_per_token_ms/adv:0.0004067788792366141 - perf/total_num_tokens:387035 - perf/time_per_step:67.497363332659 - perf/throughput:716.7594793527489 (TaskRunner pid=2281424) Training Progress: 51%|█████ | 51/100 [59:03<1:28:37, 108.52s/it] (TaskRunner pid=2281424) step:52 - global_seqlen/min:38872 - global_seqlen/max:61135 - global_seqlen/minmax_diff:22263 - global_seqlen/balanced_min:50883 - global_seqlen/balanced_max:50962 - global_seqlen/mean:50922.75 - actor/entropy:0.4251244068145752 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31729912757873535 - training/rollout_probs_diff_mean:0.0034149810671806335 - training/rollout_probs_diff_std:0.008329342119395733 - training/rollout_actor_probs_pearson_corr:0.9995748996734619 - rollout_corr/rollout_is_mean:0.9999804496765137 - rollout_corr/rollout_is_ratio_fraction_high:2.9626644391100854e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.925328878220171e-06 - rollout_corr/rollout_is_max:2.0289995670318604 - rollout_corr/rollout_is_min:0.38479822874069214 - rollout_corr/rollout_is_std:0.02867795154452324 - rollout_corr/rollout_is_eff_sample_size:0.9991781319248341 - rollout_corr/rollout_is_seq_mean:1.0001072883605957 - rollout_corr/rollout_is_seq_std:0.0014209682121872902 - rollout_corr/rollout_is_seq_max:1.0089370012283325 - rollout_corr/rollout_is_seq_min:0.9960955381393433 - rollout_corr/rollout_is_seq_max_deviation:0.00893700122833252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4353607962839305) - rollout_corr/training_log_ppl:np.float64(0.34465374518185854) - rollout_corr/kl:np.float64(0.00030015603282895764) - rollout_corr/k3_kl:np.float64(0.00044390009770722827) - rollout_corr/rollout_ppl:np.float64(1.4348906846717) - rollout_corr/rollout_log_ppl:np.float64(0.3443535875703674) - rollout_corr/log_ppl_diff:np.float64(0.0003001576114911586) - rollout_corr/log_ppl_abs_diff:np.float64(0.000979981996351853) - rollout_corr/log_ppl_diff_max:np.float64(0.006529062986373901) - rollout_corr/log_ppl_diff_min:np.float64(-0.006823122501373291) - rollout_corr/ppl_ratio:np.float64(1.0003011787775904) - rollout_corr/chi2_token:np.float64(0.0011839414946734905) - rollout_corr/chi2_seq:np.float64(2.5429790497291833) - actor/pg_loss:np.float64(3.4721335396170616e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.84185026279738e-05) - actor/ppo_kl:np.float64(-4.0347963830456024e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11110754683613777) - perf/mfu/actor:np.float64(0.16599907427815286) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.80552291870117) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004099216777831316 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004099216777831316 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1318.4921875 - response_length/max:5029.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1318.4921875 - response_length_non_aborted/max:5029.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001096632331609726 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7103512845933437) - timing_s/agent_loop/generate_sequences/max:np.float64(38.38058450073004) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.082279307935096) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.38058450073004) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:365 - timing_s/agent_loop/slowest/response_length:4841 - timing_s/gen:45.04192319884896 - timing_s/reward:0.10605371557176113 - timing_s/old_log_prob:4.176968155428767 - timing_s/adv:0.13944151997566223 - timing_s/update_actor:16.25410868227482 - timing_s/step:65.81346314027905 - timing_s/stop_profile:0.00014881044626235962 - timing_per_token_ms/gen:0.13344410696062903 - timing_per_token_ms/update_actor:0.03989893682655301 - timing_per_token_ms/adv:0.00034228689528664064 - perf/total_num_tokens:407382 - perf/time_per_step:65.81346314027905 - perf/throughput:773.7436623181486 (TaskRunner pid=2281424) Training Progress: 52%|█████▏ | 52/100 [1:00:09<1:16:34, 95.73s/it] (TaskRunner pid=2281424) step:53 - global_seqlen/min:24839 - global_seqlen/max:55902 - global_seqlen/minmax_diff:31063 - global_seqlen/balanced_min:43329 - global_seqlen/balanced_max:43584 - global_seqlen/mean:43520.375 - actor/entropy:0.37126773595809937 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016398251056671 - training/rollout_probs_diff_mean:0.003143634181469679 - training/rollout_probs_diff_std:0.00820545107126236 - training/rollout_actor_probs_pearson_corr:0.9994932413101196 - rollout_corr/rollout_is_mean:0.9999392628669739 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.210143394331681e-06 - rollout_corr/rollout_is_max:1.7527995109558105 - rollout_corr/rollout_is_min:0.30146440863609314 - rollout_corr/rollout_is_std:0.027419937774538994 - rollout_corr/rollout_is_eff_sample_size:0.9992485928388086 - rollout_corr/rollout_is_seq_mean:0.9999774694442749 - rollout_corr/rollout_is_seq_std:0.0011340720811858773 - rollout_corr/rollout_is_seq_max:1.005629062652588 - rollout_corr/rollout_is_seq_min:0.9950371980667114 - rollout_corr/rollout_is_seq_max_deviation:0.005629062652587891 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3701058248989284) - rollout_corr/training_log_ppl:np.float64(0.2918059763032943) - rollout_corr/kl:np.float64(0.0003194374729091898) - rollout_corr/k3_kl:np.float64(0.00042779503252177165) - rollout_corr/rollout_ppl:np.float64(1.3696815236471593) - rollout_corr/rollout_log_ppl:np.float64(0.29148653568699956) - rollout_corr/log_ppl_diff:np.float64(0.00031944061629474163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009462947491556406) - rollout_corr/log_ppl_diff_max:np.float64(0.006018996238708496) - rollout_corr/log_ppl_diff_min:np.float64(-0.003216482698917389) - rollout_corr/ppl_ratio:np.float64(1.0003202487714589) - rollout_corr/chi2_token:np.float64(0.0010781094897538424) - rollout_corr/chi2_seq:np.float64(1.596792209893465) - actor/pg_loss:np.float64(3.835768438875675e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021167921056530759) - actor/ppo_kl:np.float64(-7.656292218705119e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13869349658489227) - perf/mfu/actor:np.float64(0.14950320003636675) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.62938690185547) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015568501316010952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015568501316010952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1083.54296875 - response_length/max:5024.0 - response_length/min:193.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1083.54296875 - response_length_non_aborted/max:5024.0 - response_length_non_aborted/min:193.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011430680751800537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2967226803302765) - timing_s/agent_loop/generate_sequences/max:np.float64(36.389772513881326) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.762310426936892) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.389772513881326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:5024 - timing_s/gen:43.48752962425351 - timing_s/reward:0.08610256388783455 - timing_s/old_log_prob:3.7902569361031055 - timing_s/adv:0.1312139704823494 - timing_s/update_actor:15.44841111637652 - timing_s/step:63.040359219536185 - timing_s/stop_profile:0.00011725910007953644 - timing_per_token_ms/gen:0.15677565864389287 - timing_per_token_ms/update_actor:0.04437120290317041 - timing_per_token_ms/adv:0.0003768751144789923 - perf/total_num_tokens:348163 - perf/time_per_step:63.040359219536185 - perf/throughput:690.3573447042328 (TaskRunner pid=2281424) Training Progress: 53%|█████▎ | 53/100 [1:01:12<1:07:19, 85.94s/it] (TaskRunner pid=2281424) step:54 - global_seqlen/min:35066 - global_seqlen/max:65296 - global_seqlen/minmax_diff:30230 - global_seqlen/balanced_min:48016 - global_seqlen/balanced_max:48715 - global_seqlen/mean:48320.375 - actor/entropy:0.41513416171073914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3469814658164978 - training/rollout_probs_diff_mean:0.0032787041272968054 - training/rollout_probs_diff_std:0.008095493540167809 - training/rollout_actor_probs_pearson_corr:0.999546229839325 - rollout_corr/rollout_is_mean:0.9998549818992615 - rollout_corr/rollout_is_ratio_fraction_high:3.120212340945727e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.360637704958208e-06 - rollout_corr/rollout_is_max:3.656163454055786 - rollout_corr/rollout_is_min:0.04287761077284813 - rollout_corr/rollout_is_std:0.027915427461266518 - rollout_corr/rollout_is_eff_sample_size:0.99922121664114 - rollout_corr/rollout_is_seq_mean:0.9998648166656494 - rollout_corr/rollout_is_seq_std:0.0012277925852686167 - rollout_corr/rollout_is_seq_max:1.0036733150482178 - rollout_corr/rollout_is_seq_min:0.9944618940353394 - rollout_corr/rollout_is_seq_max_deviation:0.0055381059646606445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.386646915692836) - rollout_corr/training_log_ppl:np.float64(0.31141754146665335) - rollout_corr/kl:np.float64(0.00046588320381735215) - rollout_corr/k3_kl:np.float64(0.00041930354774422085) - rollout_corr/rollout_ppl:np.float64(1.3860031738877296) - rollout_corr/rollout_log_ppl:np.float64(0.31095165672013536) - rollout_corr/log_ppl_diff:np.float64(0.00046588474651798606) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009574951836839318) - rollout_corr/log_ppl_diff_max:np.float64(0.0065114498138427734) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028219521045684814) - rollout_corr/ppl_ratio:np.float64(1.0004668282344937) - rollout_corr/chi2_token:np.float64(0.000739572336897254) - rollout_corr/chi2_seq:np.float64(1.1254641045816243) - actor/pg_loss:np.float64(-1.3741431757807732e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.72049774898187e-05) - actor/ppo_kl:np.float64(-8.353029136642931e-05) - actor/pg_clipfrac_lower:np.float64(1.2657970955842757e-06) - actor/grad_norm:np.float64(0.09051301795989275) - perf/mfu/actor:np.float64(0.15693746638014805) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.6057014465332) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015124449506402016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015124449506402016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1251.91796875 - response_length/max:8192.0 - response_length/min:151.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1251.91796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.027255535125732e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3670800887048244) - timing_s/agent_loop/generate_sequences/max:np.float64(57.39985219575465) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.685668693418847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.39985219575465) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.27751941606402 - timing_s/reward:0.09590023942291737 - timing_s/old_log_prob:4.017355827614665 - timing_s/adv:0.11962731927633286 - timing_s/update_actor:16.561819057911634 - timing_s/step:85.16014346294105 - timing_s/stop_profile:0.00010659545660018921 - timing_per_token_ms/gen:0.20055951466987848 - timing_per_token_ms/update_actor:0.04284377723142575 - timing_per_token_ms/adv:0.0003094639664849788 - perf/total_num_tokens:386563 - perf/time_per_step:85.16014346294105 - perf/throughput:567.4059840097318 (TaskRunner pid=2281424) Training Progress: 54%|█████▍ | 54/100 [1:02:38<1:05:43, 85.72s/it] (TaskRunner pid=2281424) step:55 - global_seqlen/min:28028 - global_seqlen/max:53093 - global_seqlen/minmax_diff:25065 - global_seqlen/balanced_min:38416 - global_seqlen/balanced_max:38451 - global_seqlen/mean:38438.25 - actor/entropy:0.3102551996707916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5002540349960327 - training/rollout_probs_diff_mean:0.0031136018224060535 - training/rollout_probs_diff_std:0.008566453121602535 - training/rollout_actor_probs_pearson_corr:0.9994353652000427 - rollout_corr/rollout_is_mean:0.9999341368675232 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.173030447418569e-06 - rollout_corr/rollout_is_max:1.8234162330627441 - rollout_corr/rollout_is_min:0.46666446328163147 - rollout_corr/rollout_is_std:0.02762567438185215 - rollout_corr/rollout_is_eff_sample_size:0.9992372850970654 - rollout_corr/rollout_is_seq_mean:0.999902606010437 - rollout_corr/rollout_is_seq_std:0.0013777980348095298 - rollout_corr/rollout_is_seq_max:1.0075701475143433 - rollout_corr/rollout_is_seq_min:0.9948606491088867 - rollout_corr/rollout_is_seq_max_deviation:0.007570147514343262 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.308470833580941) - rollout_corr/training_log_ppl:np.float64(0.2564443113806192) - rollout_corr/kl:np.float64(0.0004685787036251554) - rollout_corr/k3_kl:np.float64(0.0005076911484707125) - rollout_corr/rollout_ppl:np.float64(1.3078627600334585) - rollout_corr/rollout_log_ppl:np.float64(0.255975730920909) - rollout_corr/log_ppl_diff:np.float64(0.00046858045971021056) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011701608309522271) - rollout_corr/log_ppl_diff_max:np.float64(0.006679356098175049) - rollout_corr/log_ppl_diff_min:np.float64(-0.0075002312660217285) - rollout_corr/ppl_ratio:np.float64(1.000470134196803) - rollout_corr/chi2_token:np.float64(0.0011455377098172903) - rollout_corr/chi2_seq:np.float64(5.852095796493813) - actor/pg_loss:np.float64(-3.334670327603817e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001705758450043504) - actor/ppo_kl:np.float64(-4.5051564853793025e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07974306493997574) - perf/mfu/actor:np.float64(0.1378203781579895) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.55471420288086) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011153988540172577 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011153988540172577 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:936.0703125 - response_length/max:3898.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:936.0703125 - response_length_non_aborted/max:3898.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001405421644449234 - timing_s/agent_loop/generate_sequences/min:np.float64(1.978264905512333) - timing_s/agent_loop/generate_sequences/max:np.float64(29.77366792783141) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.763206802737841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.77366792783141) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:3898 - timing_s/gen:36.85863957554102 - timing_s/reward:0.08288105763494968 - timing_s/old_log_prob:3.500804428011179 - timing_s/adv:0.11686178669333458 - timing_s/update_actor:14.765768744051456 - timing_s/step:55.41219947300851 - timing_s/stop_profile:0.00012343376874923706 - timing_per_token_ms/gen:0.1538122285466212 - timing_per_token_ms/update_actor:0.04801782321012096 - timing_per_token_ms/adv:0.0003800309154726561 - perf/total_num_tokens:307506 - perf/time_per_step:55.41219947300851 - perf/throughput:693.6784745157682 (TaskRunner pid=2281424) Training Progress: 55%|█████▌ | 55/100 [1:03:33<57:28, 76.64s/it] (TaskRunner pid=2281424) step:56 - global_seqlen/min:35222 - global_seqlen/max:67738 - global_seqlen/minmax_diff:32516 - global_seqlen/balanced_min:48431 - global_seqlen/balanced_max:48616 - global_seqlen/mean:48556.375 - actor/entropy:0.3693101406097412 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5576332211494446 - training/rollout_probs_diff_mean:0.003098106477409601 - training/rollout_probs_diff_std:0.008025160990655422 - training/rollout_actor_probs_pearson_corr:0.9995522499084473 - rollout_corr/rollout_is_mean:0.999983012676239 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.090129894189886e-06 - rollout_corr/rollout_is_max:1.6744037866592407 - rollout_corr/rollout_is_min:0.3452942669391632 - rollout_corr/rollout_is_std:0.02711610123515129 - rollout_corr/rollout_is_eff_sample_size:0.9992651383115954 - rollout_corr/rollout_is_seq_mean:0.9999608993530273 - rollout_corr/rollout_is_seq_std:0.0010731675429269671 - rollout_corr/rollout_is_seq_max:1.006026029586792 - rollout_corr/rollout_is_seq_min:0.9958376288414001 - rollout_corr/rollout_is_seq_max_deviation:0.006026029586791992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.373812660574913) - rollout_corr/training_log_ppl:np.float64(0.3022938699577935) - rollout_corr/kl:np.float64(0.0004166627330413064) - rollout_corr/k3_kl:np.float64(0.00039845293167672935) - rollout_corr/rollout_ppl:np.float64(1.3732107440009713) - rollout_corr/rollout_log_ppl:np.float64(0.3018772063078359) - rollout_corr/log_ppl_diff:np.float64(0.00041666364995762706) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008859913214109838) - rollout_corr/log_ppl_diff_max:np.float64(0.00528949499130249) - rollout_corr/log_ppl_diff_min:np.float64(-0.005298733711242676) - rollout_corr/ppl_ratio:np.float64(1.0004174057394266) - rollout_corr/chi2_token:np.float64(0.0007626630831509829) - rollout_corr/chi2_seq:np.float64(3.9297358801122755) - actor/pg_loss:np.float64(-1.1299271136522293e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014216022026403152) - actor/ppo_kl:np.float64(-9.481449527726227e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08582276105880737) - perf/mfu/actor:np.float64(0.15478688954187692) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(117.41515350341797) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01221644226461649 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.01221644226461649 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:1264.10546875 - response_length/max:5770.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1264.10546875 - response_length_non_aborted/max:5770.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013057328760623932 - timing_s/agent_loop/generate_sequences/min:np.float64(2.215219648554921) - timing_s/agent_loop/generate_sequences/max:np.float64(45.5247247479856) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.946847598039312) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(45.5247247479856) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:5770 - timing_s/gen:52.53792614676058 - timing_s/reward:0.09584798477590084 - timing_s/old_log_prob:3.963084263727069 - timing_s/adv:0.13674898445606232 - timing_s/update_actor:16.772162212058902 - timing_s/step:73.59324477799237 - timing_s/stop_profile:3.3739954233169556e-05 - timing_per_token_ms/gen:0.1623490120754875 - timing_per_token_ms/update_actor:0.04317703445752206 - timing_per_token_ms/adv:0.0003520366389996739 - perf/total_num_tokens:388451 - perf/time_per_step:73.59324477799237 - perf/throughput:659.7939137821587 (TaskRunner pid=2281424) Training Progress: 56%|█████▌ | 56/100 [1:04:47<55:32, 75.74s/it] (TaskRunner pid=2281424) step:57 - global_seqlen/min:50111 - global_seqlen/max:61804 - global_seqlen/minmax_diff:11693 - global_seqlen/balanced_min:54058 - global_seqlen/balanced_max:54380 - global_seqlen/mean:54299.125 - actor/entropy:0.3828109800815582 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26735612750053406 - training/rollout_probs_diff_mean:0.003163557732477784 - training/rollout_probs_diff_std:0.007935766130685806 - training/rollout_actor_probs_pearson_corr:0.9995775818824768 - rollout_corr/rollout_is_mean:0.9999682903289795 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.757548145382316e-06 - rollout_corr/rollout_is_max:1.6177020072937012 - rollout_corr/rollout_is_min:0.4519280195236206 - rollout_corr/rollout_is_std:0.027300119400024414 - rollout_corr/rollout_is_eff_sample_size:0.9992552585748898 - rollout_corr/rollout_is_seq_mean:0.9999948740005493 - rollout_corr/rollout_is_seq_std:0.0008971233037300408 - rollout_corr/rollout_is_seq_max:1.004016637802124 - rollout_corr/rollout_is_seq_min:0.9975495934486389 - rollout_corr/rollout_is_seq_max_deviation:0.0040166378021240234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4292010506615043) - rollout_corr/training_log_ppl:np.float64(0.3405683948658407) - rollout_corr/kl:np.float64(0.00034688604483079644) - rollout_corr/k3_kl:np.float64(0.00040535901956673115) - rollout_corr/rollout_ppl:np.float64(1.4287002789787948) - rollout_corr/rollout_log_ppl:np.float64(0.34022150505916215) - rollout_corr/log_ppl_diff:np.float64(0.00034688980667851865) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007467021641787142) - rollout_corr/log_ppl_diff_max:np.float64(0.0035548806190490723) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031991004943847656) - rollout_corr/ppl_ratio:np.float64(1.0003474049735814) - rollout_corr/chi2_token:np.float64(0.0009404013399034739) - rollout_corr/chi2_seq:np.float64(1.3481947318650782) - actor/pg_loss:np.float64(-1.1662370525300503e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001704909616933037) - actor/ppo_kl:np.float64(-4.2620445977803456e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11611825972795486) - perf/mfu/actor:np.float64(0.13975899720323387) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(142.28152465820312) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023128935135900974 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023128935135900974 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1416.56640625 - response_length/max:6110.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1416.56640625 - response_length_non_aborted/max:6110.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.9925798773765564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7666615955531597) - timing_s/agent_loop/generate_sequences/max:np.float64(44.93153551965952) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.538133056579682) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.93153551965952) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:382 - timing_s/agent_loop/slowest/response_length:6110 - timing_s/gen:52.147955069318414 - timing_s/reward:0.10221655108034611 - timing_s/old_log_prob:4.09379106014967 - timing_s/adv:0.1320952270179987 - timing_s/update_actor:20.559559671208262 - timing_s/step:77.12406794913113 - timing_s/stop_profile:0.00012364424765110016 - timing_per_token_ms/gen:0.14380049434376813 - timing_per_token_ms/update_actor:0.04732939911832894 - timing_per_token_ms/adv:0.0003040915185511707 - perf/total_num_tokens:434393 - perf/time_per_step:77.12406794913113 - perf/throughput:704.0490270276482 (TaskRunner pid=2281424) Training Progress: 57%|█████▋ | 57/100 [1:06:04<54:35, 76.17s/it] (TaskRunner pid=2281424) step:58 - global_seqlen/min:26330 - global_seqlen/max:76091 - global_seqlen/minmax_diff:49761 - global_seqlen/balanced_min:58973 - global_seqlen/balanced_max:59329 - global_seqlen/mean:59224.75 - actor/entropy:0.41788530349731445 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2639843225479126 - training/rollout_probs_diff_mean:0.0031849679071456194 - training/rollout_probs_diff_std:0.007847386412322521 - training/rollout_actor_probs_pearson_corr:0.9995084404945374 - rollout_corr/rollout_is_mean:1.0000232458114624 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4740838853176683e-06 - rollout_corr/rollout_is_max:1.8915661573410034 - rollout_corr/rollout_is_min:0.4832708239555359 - rollout_corr/rollout_is_std:0.027686020359396935 - rollout_corr/rollout_is_eff_sample_size:0.9992339523378033 - rollout_corr/rollout_is_seq_mean:0.999971866607666 - rollout_corr/rollout_is_seq_std:0.001041158102452755 - rollout_corr/rollout_is_seq_max:1.0040910243988037 - rollout_corr/rollout_is_seq_min:0.9945273399353027 - rollout_corr/rollout_is_seq_max_deviation:0.005472660064697266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4465448674745858) - rollout_corr/training_log_ppl:np.float64(0.35632477820036) - rollout_corr/kl:np.float64(0.0003769699203459709) - rollout_corr/k3_kl:np.float64(0.0005920725309920272) - rollout_corr/rollout_ppl:np.float64(1.4459977843798697) - rollout_corr/rollout_log_ppl:np.float64(0.35594780783867463) - rollout_corr/log_ppl_diff:np.float64(0.00037697036168538034) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009633789595682174) - rollout_corr/log_ppl_diff_max:np.float64(0.006280913949012756) - rollout_corr/log_ppl_diff_min:np.float64(-0.004786491394042969) - rollout_corr/ppl_ratio:np.float64(1.000377933960408) - rollout_corr/chi2_token:np.float64(0.0016758001875132322) - rollout_corr/chi2_seq:np.float64(7.071388645563275) - actor/pg_loss:np.float64(1.6489182598888874e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025524779363195194) - actor/ppo_kl:np.float64(-7.475169755899458e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11395814083516598) - perf/mfu/actor:np.float64(0.17468793390598217) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(112.19358444213867) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017182203009724617 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017182203009724617 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1578.8671875 - response_length/max:6944.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1578.8671875 - response_length_non_aborted/max:6944.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018575415015220642 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6479607988148928) - timing_s/agent_loop/generate_sequences/max:np.float64(53.533365273848176) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.731221175505198) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(53.533365273848176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:6944 - timing_s/gen:60.268137166276574 - timing_s/reward:0.11793672852218151 - timing_s/old_log_prob:4.338053205981851 - timing_s/adv:0.14384562522172928 - timing_s/update_actor:18.135621577501297 - timing_s/step:83.09424239955842 - timing_s/stop_profile:0.00011111609637737274 - timing_per_token_ms/gen:0.14910843208955338 - timing_per_token_ms/update_actor:0.03827711720501416 - timing_per_token_ms/adv:0.0003036011659435651 - perf/total_num_tokens:473798 - perf/time_per_step:83.09424239955842 - perf/throughput:712.7419215788498 (TaskRunner pid=2281424) Training Progress: 58%|█████▊ | 58/100 [1:07:27<54:46, 78.26s/it] (TaskRunner pid=2281424) step:59 - global_seqlen/min:34689 - global_seqlen/max:56180 - global_seqlen/minmax_diff:21491 - global_seqlen/balanced_min:47141 - global_seqlen/balanced_max:49075 - global_seqlen/mean:47480.625 - actor/entropy:0.42323192954063416 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22112640738487244 - training/rollout_probs_diff_mean:0.0033911745995283127 - training/rollout_probs_diff_std:0.008138471283018589 - training/rollout_actor_probs_pearson_corr:0.9995591044425964 - rollout_corr/rollout_is_mean:0.9999943971633911 - rollout_corr/rollout_is_ratio_fraction_high:3.213068112017936e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.426136224035872e-06 - rollout_corr/rollout_is_max:2.264416217803955 - rollout_corr/rollout_is_min:0.44598260521888733 - rollout_corr/rollout_is_std:0.02836867794394493 - rollout_corr/rollout_is_eff_sample_size:0.9991957462217225 - rollout_corr/rollout_is_seq_mean:0.9999998211860657 - rollout_corr/rollout_is_seq_std:0.001030960469506681 - rollout_corr/rollout_is_seq_max:1.004229187965393 - rollout_corr/rollout_is_seq_min:0.9960595369338989 - rollout_corr/rollout_is_seq_max_deviation:0.004229187965393066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4299974106252193) - rollout_corr/training_log_ppl:np.float64(0.3359124011185486) - rollout_corr/kl:np.float64(0.00043199253835979334) - rollout_corr/k3_kl:np.float64(0.0004072271172077535) - rollout_corr/rollout_ppl:np.float64(1.429351010825485) - rollout_corr/rollout_log_ppl:np.float64(0.3354804104164941) - rollout_corr/log_ppl_diff:np.float64(0.0004319907020544633) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008698585879756138) - rollout_corr/log_ppl_diff_max:np.float64(0.0032856017351150513) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038261711597442627) - rollout_corr/ppl_ratio:np.float64(1.0004326105117798) - rollout_corr/chi2_token:np.float64(0.0007886665407568216) - rollout_corr/chi2_seq:np.float64(1.102270802250132) - actor/pg_loss:np.float64(5.7405210100114346e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012484539280421814) - actor/ppo_kl:np.float64(4.3178232203011646e-05) - actor/pg_clipfrac_lower:np.float64(2.1677303720935015e-06) - actor/grad_norm:np.float64(0.12459838949143887) - perf/mfu/actor:np.float64(0.15411530957231614) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.97130966186523) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04919448494911194 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04919448494911194 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1215.73828125 - response_length/max:7645.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1215.73828125 - response_length_non_aborted/max:7645.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017377547919750214 - timing_s/agent_loop/generate_sequences/min:np.float64(2.034050405025482) - timing_s/agent_loop/generate_sequences/max:np.float64(52.7514441665262) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.213702496665064) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(52.7514441665262) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:7645 - timing_s/gen:60.28812167607248 - timing_s/reward:0.09481470286846161 - timing_s/old_log_prob:3.9277980905026197 - timing_s/adv:0.1367920432239771 - timing_s/update_actor:16.420462617650628 - timing_s/step:80.95495403930545 - timing_s/stop_profile:0.00012053921818733215 - timing_per_token_ms/gen:0.19370984604928357 - timing_per_token_ms/update_actor:0.04322937676591933 - timing_per_token_ms/adv:0.00036012595459720964 - perf/total_num_tokens:379845 - perf/time_per_step:80.95495403930545 - perf/throughput:586.5067254185221 (TaskRunner pid=2281424) Training Progress: 59%|█████▉ | 59/100 [1:08:48<54:02, 79.08s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the charge $ q $, we are given: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now substitute the known values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) First, compute $ (0.50)^2 $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (0.50)^2 = 0.25 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compute the numerator: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 2.0 \cdot 0.25 = 0.50 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compute the denominator: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 8.99 \times 10^9 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compute the charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} = \frac{0.50}{8.99} \times 10^{-9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) \frac{0.50}{8.99} \approx 0.0556 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 0.0556 \times 10^{-9} = 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now convert to picocoulombs (pC), since $ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compare with the answer choices: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest match is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula $ E = k \frac{q}{r^2} $ to find the charge $ q $. Substituting the given values and simplifying, we find that the charge is approximately 55.6 pC, which is closest to option B. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2281424) step:60 - global_seqlen/min:30111 - global_seqlen/max:89796 - global_seqlen/minmax_diff:59685 - global_seqlen/balanced_min:50047 - global_seqlen/balanced_max:50180 - global_seqlen/mean:50131.125 - actor/entropy:0.4219725430011749 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26761937141418457 - training/rollout_probs_diff_mean:0.0032737525179982185 - training/rollout_probs_diff_std:0.008023563772439957 - training/rollout_actor_probs_pearson_corr:0.9995866417884827 - rollout_corr/rollout_is_mean:0.9999962449073792 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.0261113503656816e-06 - rollout_corr/rollout_is_max:1.9552534818649292 - rollout_corr/rollout_is_min:0.36368343234062195 - rollout_corr/rollout_is_std:0.028026238083839417 - rollout_corr/rollout_is_eff_sample_size:0.999215146470059 - rollout_corr/rollout_is_seq_mean:0.9999516606330872 - rollout_corr/rollout_is_seq_std:0.0010303977178409696 - rollout_corr/rollout_is_seq_max:1.0035020112991333 - rollout_corr/rollout_is_seq_min:0.9954249262809753 - rollout_corr/rollout_is_seq_max_deviation:0.004575073719024658 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4140595034696162) - rollout_corr/training_log_ppl:np.float64(0.3272325733851176) - rollout_corr/kl:np.float64(0.00043046686714831583) - rollout_corr/k3_kl:np.float64(0.0003949974453973937) - rollout_corr/rollout_ppl:np.float64(1.4134283671155572) - rollout_corr/rollout_log_ppl:np.float64(0.32680210378021) - rollout_corr/log_ppl_diff:np.float64(0.00043046960490755737) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008739173354115337) - rollout_corr/log_ppl_diff_max:np.float64(0.006730854511260986) - rollout_corr/log_ppl_diff_min:np.float64(-0.002438843250274658) - rollout_corr/ppl_ratio:np.float64(1.0004312125965953) - rollout_corr/chi2_token:np.float64(0.000722584780305624) - rollout_corr/chi2_seq:np.float64(1.0466190420556813) - actor/pg_loss:np.float64(1.1199153959751129e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015968021341450367) - actor/ppo_kl:np.float64(6.704961512227214e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12008443288505077) - perf/mfu/actor:np.float64(0.1647866551242615) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.77543354034424) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.671875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2240626490983253) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7666999999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1971325056706902) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5781125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18312620134038018) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.671675) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2244978550800715) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8490500000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14877363384575418) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5088) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1262359553956552) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6999125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.19120944443302226) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.9071374999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09393121099917975) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.46338749999999995) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08209148087342806) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7191875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15548933759230438) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9402375) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.050091787700358235) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4305) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05314562557979722) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7322124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.12204692054822548) - val-aux/sciknoweval/score/mean@16:np.float64(0.671875) - val-aux/sciknoweval/score/std@16:np.float64(0.2240626490983253) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7666999999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1971325056706902) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5781125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18312620134038018) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.671675) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2244978550800715) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8490500000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14877363384575418) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5088) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1262359553956552) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6999125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.19120944443302226) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.9071374999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09393121099917975) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.46338749999999995) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08209148087342806) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7191875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15548933759230438) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9402375) - val-aux/sciknoweval/score/best@16/std:np.float64(0.050091787700358235) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4305) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05314562557979722) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7322124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.12204692054822548) - val-core/sciknoweval/acc/mean@16:np.float64(0.671875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2240626490983253) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7666999999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1971325056706902) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5781125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18312620134038018) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.671675) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2244978550800715) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8490500000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14877363384575418) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5088) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1262359553956552) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6999125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.19120944443302226) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.9071374999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09393121099917975) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.46338749999999995) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08209148087342806) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7191875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15548933759230438) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9402375) - val-core/sciknoweval/acc/best@16/std:np.float64(0.050091787700358235) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4305) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05314562557979722) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7322124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.12204692054822548) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.95078125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.09649250755143636) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.98285) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.043210438568887635) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9181125) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11503551772140294) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9507125000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.09563050360637051) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9944124999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.014654216053618907) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8752125) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12749716856579513) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9671624999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0726457037962381) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9986749999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0056780934585833245) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8206) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.12838320809724896) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9803000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.047900368785141687) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.001243734296383275) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7596999999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.10958385062712987) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9878125000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.028513078887782435) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015586912631988525 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015586912631988525 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1296.44140625 - response_length/max:4897.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1296.44140625 - response_length_non_aborted/max:4897.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010100007057189941 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3318876773118973) - timing_s/agent_loop/generate_sequences/max:np.float64(37.898281663656235) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.311515245914052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(37.898281663656235) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:4897 - timing_s/gen:44.70199620164931 - timing_s/reward:0.09790181741118431 - timing_s/old_log_prob:3.9458488430827856 - timing_s/adv:0.12571285665035248 - timing_s/update_actor:16.154612444341183 - timing_s/step:65.11406415887177 - timing_s/testing:175.5087940171361 - timing_s/save_checkpoint:12.78239992260933 - timing_s/stop_profile:0.00016752071678638458 - timing_per_token_ms/gen:0.13468959863583702 - timing_per_token_ms/update_actor:0.040280894465118185 - timing_per_token_ms/adv:0.00031346009253321286 - perf/total_num_tokens:401049 - perf/time_per_step:65.11406415887177 - perf/throughput:769.8970360333383 (TaskRunner pid=2281424) Training Progress: 60%|██████ | 60/100 [1:13:01<1:27:36, 131.40s/it] (TaskRunner pid=2281424) step:61 - global_seqlen/min:45752 - global_seqlen/max:77521 - global_seqlen/minmax_diff:31769 - global_seqlen/balanced_min:56607 - global_seqlen/balanced_max:56958 - global_seqlen/mean:56856.125 - actor/entropy:0.39492928981781006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6552004814147949 - training/rollout_probs_diff_mean:0.003135534469038248 - training/rollout_probs_diff_std:0.007907739840447903 - training/rollout_actor_probs_pearson_corr:0.9994683861732483 - rollout_corr/rollout_is_mean:0.9999828338623047 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.567888486737502e-06 - rollout_corr/rollout_is_max:1.8672653436660767 - rollout_corr/rollout_is_min:0.29101717472076416 - rollout_corr/rollout_is_std:0.027300119400024414 - rollout_corr/rollout_is_eff_sample_size:0.9992550205113556 - rollout_corr/rollout_is_seq_mean:1.0000195503234863 - rollout_corr/rollout_is_seq_std:0.0009980873437598348 - rollout_corr/rollout_is_seq_max:1.0043768882751465 - rollout_corr/rollout_is_seq_min:0.9967591166496277 - rollout_corr/rollout_is_seq_max_deviation:0.004376888275146484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.408340530935675) - rollout_corr/training_log_ppl:np.float64(0.324529722915031) - rollout_corr/kl:np.float64(0.00038801542386091015) - rollout_corr/k3_kl:np.float64(0.00042000259992391875) - rollout_corr/rollout_ppl:np.float64(1.407751803752035) - rollout_corr/rollout_log_ppl:np.float64(0.32414170513220597) - rollout_corr/log_ppl_diff:np.float64(0.0003880177828250453) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008422878890996799) - rollout_corr/log_ppl_diff_max:np.float64(0.005800724029541016) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036716610193252563) - rollout_corr/ppl_ratio:np.float64(1.0003886730410159) - rollout_corr/chi2_token:np.float64(0.0008997754193842411) - rollout_corr/chi2_seq:np.float64(0.5856562717817724) - actor/pg_loss:np.float64(9.93476714938879e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020609253107295444) - actor/ppo_kl:np.float64(1.0061916269954452e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15778088383376598) - perf/mfu/actor:np.float64(0.17557393231026636) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.95418548583984) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009713038452900946 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009713038452900946 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1521.19140625 - response_length/max:6659.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1521.19140625 - response_length_non_aborted/max:6659.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001254584640264511 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8240225315093994) - timing_s/agent_loop/generate_sequences/max:np.float64(49.38348316960037) - timing_s/agent_loop/generate_sequences/mean:np.float64(15.205760123368236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.38348316960037) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:6659 - timing_s/gen:55.99783328920603 - timing_s/reward:0.10260644741356373 - timing_s/old_log_prob:4.418327359482646 - timing_s/adv:0.13144686073064804 - timing_s/update_actor:17.245223624631763 - timing_s/step:77.9999400805682 - timing_s/stop_profile:0.00013169646263122559 - timing_per_token_ms/gen:0.14379619513181235 - timing_per_token_ms/update_actor:0.03791417288953425 - timing_per_token_ms/adv:0.00028899010601462913 - perf/total_num_tokens:454849 - perf/time_per_step:77.9999400805682 - perf/throughput:728.9252394459767 (TaskRunner pid=2281424) Training Progress: 61%|██████ | 61/100 [1:14:19<1:15:00, 115.40s/it] (TaskRunner pid=2281424) step:62 - global_seqlen/min:28896 - global_seqlen/max:104683 - global_seqlen/minmax_diff:75787 - global_seqlen/balanced_min:67049 - global_seqlen/balanced_max:67213 - global_seqlen/mean:67134.0 - actor/entropy:0.42600366473197937 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8567732572555542 - training/rollout_probs_diff_mean:0.0032150347251445055 - training/rollout_probs_diff_std:0.00793883204460144 - training/rollout_actor_probs_pearson_corr:0.9996106028556824 - rollout_corr/rollout_is_mean:1.0000700950622559 - rollout_corr/rollout_is_ratio_fraction_high:2.15829822991509e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.47489468974527e-06 - rollout_corr/rollout_is_max:2.125422954559326 - rollout_corr/rollout_is_min:0.003053532214835286 - rollout_corr/rollout_is_std:0.027791311964392662 - rollout_corr/rollout_is_eff_sample_size:0.9992283581133058 - rollout_corr/rollout_is_seq_mean:1.0000699758529663 - rollout_corr/rollout_is_seq_std:0.001128942472860217 - rollout_corr/rollout_is_seq_max:1.0039699077606201 - rollout_corr/rollout_is_seq_min:0.9920714497566223 - rollout_corr/rollout_is_seq_max_deviation:0.007928550243377686 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4691616958007216) - rollout_corr/training_log_ppl:np.float64(0.3653776543796994) - rollout_corr/kl:np.float64(0.00039364140611319876) - rollout_corr/k3_kl:np.float64(0.00042572843989319153) - rollout_corr/rollout_ppl:np.float64(1.4685669397003949) - rollout_corr/rollout_log_ppl:np.float64(0.36498401110293344) - rollout_corr/log_ppl_diff:np.float64(0.0003936432767659426) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008307227399200201) - rollout_corr/log_ppl_diff_max:np.float64(0.009543061256408691) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032494962215423584) - rollout_corr/ppl_ratio:np.float64(1.0003944088239223) - rollout_corr/chi2_token:np.float64(0.0009197669569402933) - rollout_corr/chi2_seq:np.float64(1.6601045140996575) - actor/pg_loss:np.float64(-9.474251419305801e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011590343615353049) - actor/ppo_kl:np.float64(4.716461066323063e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10818999446928501) - perf/mfu/actor:np.float64(0.19350604047505943) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.04900360107422) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014814019668847322 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014814019668847322 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1809.875 - response_length/max:6139.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1809.875 - response_length_non_aborted/max:6139.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.605474770069122e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1590842586010695) - timing_s/agent_loop/generate_sequences/max:np.float64(48.22148149460554) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.0422284047454) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.22148149460554) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:6139 - timing_s/gen:55.47001213952899 - timing_s/reward:0.11935079656541348 - timing_s/old_log_prob:4.546598572283983 - timing_s/adv:0.14351257495582104 - timing_s/update_actor:18.579031608998775 - timing_s/step:78.9453232511878 - timing_s/stop_profile:0.00011393986642360687 - timing_per_token_ms/gen:0.11972082874233586 - timing_per_token_ms/update_actor:0.0345931860327829 - timing_per_token_ms/adv:0.00026721291550447807 - perf/total_num_tokens:537072 - perf/time_per_step:78.9453232511878 - perf/throughput:850.386029662497 (TaskRunner pid=2281424) Training Progress: 62%|██████▏ | 62/100 [1:15:38<1:06:10, 104.48s/it] (TaskRunner pid=2281424) step:63 - global_seqlen/min:33404 - global_seqlen/max:94971 - global_seqlen/minmax_diff:61567 - global_seqlen/balanced_min:53259 - global_seqlen/balanced_max:54267 - global_seqlen/mean:53515.0 - actor/entropy:0.45275378227233887 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18870103359222412 - training/rollout_probs_diff_mean:0.00339776324108243 - training/rollout_probs_diff_std:0.008000901900231838 - training/rollout_actor_probs_pearson_corr:0.9995976686477661 - rollout_corr/rollout_is_mean:1.0000234842300415 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.779074748104904e-06 - rollout_corr/rollout_is_max:1.6216226816177368 - rollout_corr/rollout_is_min:0.3960243761539459 - rollout_corr/rollout_is_std:0.028292939066886902 - rollout_corr/rollout_is_eff_sample_size:0.9992000308743295 - rollout_corr/rollout_is_seq_mean:1.0000338554382324 - rollout_corr/rollout_is_seq_std:0.0010206034639850259 - rollout_corr/rollout_is_seq_max:1.0040816068649292 - rollout_corr/rollout_is_seq_min:0.9965076446533203 - rollout_corr/rollout_is_seq_max_deviation:0.004081606864929199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.392845130059868) - rollout_corr/training_log_ppl:np.float64(0.30986753129400313) - rollout_corr/kl:np.float64(0.00039362613700077986) - rollout_corr/k3_kl:np.float64(0.0003788294339130971) - rollout_corr/rollout_ppl:np.float64(1.3922858135774732) - rollout_corr/rollout_log_ppl:np.float64(0.30947390163782984) - rollout_corr/log_ppl_diff:np.float64(0.0003936296561732888) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008133103256113827) - rollout_corr/log_ppl_diff_max:np.float64(0.004159331321716309) - rollout_corr/log_ppl_diff_min:np.float64(-0.0025495290756225586) - rollout_corr/ppl_ratio:np.float64(1.0003942425828427) - rollout_corr/chi2_token:np.float64(0.0007350770756602287) - rollout_corr/chi2_seq:np.float64(0.4594551851041615) - actor/pg_loss:np.float64(7.723795715719461e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010548596083026496) - actor/ppo_kl:np.float64(5.877050272280271e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08551348559558392) - perf/mfu/actor:np.float64(0.16753065811491627) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.80081176757812) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03102802112698555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03102802112698555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1405.59375 - response_length/max:8192.0 - response_length/min:152.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1405.59375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017119385302066803 - timing_s/agent_loop/generate_sequences/min:np.float64(1.869371073320508) - timing_s/agent_loop/generate_sequences/max:np.float64(67.03272437117994) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.693403950666834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(67.03272437117994) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:74.319850102067 - timing_s/reward:0.1025898028165102 - timing_s/old_log_prob:4.21283820644021 - timing_s/adv:0.12729890830814838 - timing_s/update_actor:17.184931615367532 - timing_s/step:96.03458793275058 - timing_s/stop_profile:0.00012183934450149536 - timing_per_token_ms/gen:0.2065404135876381 - timing_per_token_ms/update_actor:0.04014045504850867 - timing_per_token_ms/adv:0.0002973439883867803 - perf/total_num_tokens:428120 - perf/time_per_step:96.03458793275058 - perf/throughput:557.2471455542095 (TaskRunner pid=2281424) Training Progress: 63%|██████▎ | 63/100 [1:17:15<1:02:52, 101.96s/it] (TaskRunner pid=2281424) step:64 - global_seqlen/min:30970 - global_seqlen/max:90543 - global_seqlen/minmax_diff:59573 - global_seqlen/balanced_min:59596 - global_seqlen/balanced_max:59937 - global_seqlen/mean:59830.0 - actor/entropy:0.43621382117271423 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4681967496871948 - training/rollout_probs_diff_mean:0.0032422388903796673 - training/rollout_probs_diff_std:0.007834543474018574 - training/rollout_actor_probs_pearson_corr:0.999615490436554 - rollout_corr/rollout_is_mean:1.000067949295044 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.873009402217576e-06 - rollout_corr/rollout_is_max:1.9102002382278442 - rollout_corr/rollout_is_min:0.2423228621482849 - rollout_corr/rollout_is_std:0.02796875685453415 - rollout_corr/rollout_is_eff_sample_size:0.999218360080856 - rollout_corr/rollout_is_seq_mean:1.0001271963119507 - rollout_corr/rollout_is_seq_std:0.0008784870151430368 - rollout_corr/rollout_is_seq_max:1.0027687549591064 - rollout_corr/rollout_is_seq_min:0.9961645007133484 - rollout_corr/rollout_is_seq_max_deviation:0.0038354992866516113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4255081894807518) - rollout_corr/training_log_ppl:np.float64(0.33409654538263567) - rollout_corr/kl:np.float64(0.0003069534103872229) - rollout_corr/k3_kl:np.float64(0.0004051391514963143) - rollout_corr/rollout_ppl:np.float64(1.425030098296702) - rollout_corr/rollout_log_ppl:np.float64(0.3337895911245141) - rollout_corr/log_ppl_diff:np.float64(0.0003069542581215501) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007710371864959598) - rollout_corr/log_ppl_diff_max:np.float64(0.00480864942073822) - rollout_corr/log_ppl_diff_min:np.float64(-0.002487003803253174) - rollout_corr/ppl_ratio:np.float64(1.0003075052518398) - rollout_corr/chi2_token:np.float64(0.0010234767105430365) - rollout_corr/chi2_seq:np.float64(1.0248696226626635) - actor/pg_loss:np.float64(3.9162347093224525e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00013449225230033335) - actor/ppo_kl:np.float64(7.314733183516609e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09787004068493843) - perf/mfu/actor:np.float64(0.18222994298278014) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.78503465652466) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012692666612565517 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012692666612565517 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1603.21875 - response_length/max:6504.0 - response_length/min:235.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1603.21875 - response_length_non_aborted/max:6504.0 - response_length_non_aborted/min:235.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016041100025177002 - timing_s/agent_loop/generate_sequences/min:np.float64(3.038218954578042) - timing_s/agent_loop/generate_sequences/max:np.float64(48.57417339272797) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.26644155262329) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.57417339272797) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:212 - timing_s/agent_loop/slowest/response_length:6504 - timing_s/gen:55.43104035779834 - timing_s/reward:0.1111034657806158 - timing_s/old_log_prob:4.2981742061674595 - timing_s/adv:0.1171770989894867 - timing_s/update_actor:17.636247349902987 - timing_s/step:77.67978408560157 - timing_s/stop_profile:0.00011309981346130371 - timing_per_token_ms/gen:0.1350579896833478 - timing_per_token_ms/update_actor:0.03684658062406608 - timing_per_token_ms/adv:0.0002448125919051619 - perf/total_num_tokens:478640 - perf/time_per_step:77.67978408560157 - perf/throughput:770.2132633899772 (TaskRunner pid=2281424) Training Progress: 64%|██████▍ | 64/100 [1:18:32<56:48, 94.69s/it] (TaskRunner pid=2281424) step:65 - global_seqlen/min:21582 - global_seqlen/max:63673 - global_seqlen/minmax_diff:42091 - global_seqlen/balanced_min:47500 - global_seqlen/balanced_max:47999 - global_seqlen/mean:47802.125 - actor/entropy:0.39129528403282166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21986928582191467 - training/rollout_probs_diff_mean:0.003225617576390505 - training/rollout_probs_diff_std:0.008090500719845295 - training/rollout_actor_probs_pearson_corr:0.9995739459991455 - rollout_corr/rollout_is_mean:0.9999448657035828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6904900074005127 - rollout_corr/rollout_is_min:0.5123902559280396 - rollout_corr/rollout_is_std:0.027487242594361305 - rollout_corr/rollout_is_eff_sample_size:0.9992447838866899 - rollout_corr/rollout_is_seq_mean:0.9998865127563477 - rollout_corr/rollout_is_seq_std:0.0011093486100435257 - rollout_corr/rollout_is_seq_max:1.0034520626068115 - rollout_corr/rollout_is_seq_min:0.9955258369445801 - rollout_corr/rollout_is_seq_max_deviation:0.004474163055419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3684682152234018) - rollout_corr/training_log_ppl:np.float64(0.29601700889179483) - rollout_corr/kl:np.float64(0.00043714539492256677) - rollout_corr/k3_kl:np.float64(0.00038878725348467924) - rollout_corr/rollout_ppl:np.float64(1.3678660304285586) - rollout_corr/rollout_log_ppl:np.float64(0.295579861412989) - rollout_corr/log_ppl_diff:np.float64(0.0004371474788058549) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009100331517402083) - rollout_corr/log_ppl_diff_max:np.float64(0.005311831831932068) - rollout_corr/log_ppl_diff_min:np.float64(-0.004062667489051819) - rollout_corr/ppl_ratio:np.float64(1.0004379414021969) - rollout_corr/chi2_token:np.float64(0.0006793311331421137) - rollout_corr/chi2_seq:np.float64(1.2524537781719118) - actor/pg_loss:np.float64(0.0001103521790355444) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.197616297891727e-05) - actor/ppo_kl:np.float64(-5.6069447737172595e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08994254469871521) - perf/mfu/actor:np.float64(0.15645029922787607) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.88603591918945) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.8671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006594820413738489 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006594820413738489 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1230.47265625 - response_length/max:7182.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1230.47265625 - response_length_non_aborted/max:7182.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.158067405223846e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5644865073263645) - timing_s/agent_loop/generate_sequences/max:np.float64(49.72097803838551) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.574511633109068) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.72097803838551) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:7182 - timing_s/gen:57.23068424500525 - timing_s/reward:0.09001462161540985 - timing_s/old_log_prob:3.9613787718117237 - timing_s/adv:0.12929501570761204 - timing_s/update_actor:16.40677917189896 - timing_s/step:77.87616940587759 - timing_s/stop_profile:0.0001377630978822708 - timing_per_token_ms/gen:0.18168413511387344 - timing_per_token_ms/update_actor:0.04290284995671992 - timing_per_token_ms/adv:0.00033809955024910514 - perf/total_num_tokens:382417 - perf/time_per_step:77.87616940587759 - perf/throughput:613.8222432444425 (TaskRunner pid=2281424) Training Progress: 65%|██████▌ | 65/100 [1:19:50<52:18, 89.66s/it] (TaskRunner pid=2281424) step:66 - global_seqlen/min:33934 - global_seqlen/max:97523 - global_seqlen/minmax_diff:63589 - global_seqlen/balanced_min:63235 - global_seqlen/balanced_max:63618 - global_seqlen/mean:63460.5 - actor/entropy:0.4836350083351135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38506191968917847 - training/rollout_probs_diff_mean:0.00350581924431026 - training/rollout_probs_diff_std:0.00794005673378706 - training/rollout_actor_probs_pearson_corr:0.9995872378349304 - rollout_corr/rollout_is_mean:0.9999473094940186 - rollout_corr/rollout_is_ratio_fraction_high:2.293304532940965e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.293304532940965e-06 - rollout_corr/rollout_is_max:3.7363619804382324 - rollout_corr/rollout_is_min:0.45649924874305725 - rollout_corr/rollout_is_std:0.02884167805314064 - rollout_corr/rollout_is_eff_sample_size:0.9991688489642058 - rollout_corr/rollout_is_seq_mean:0.9999408721923828 - rollout_corr/rollout_is_seq_std:0.0010089804418385029 - rollout_corr/rollout_is_seq_max:1.0059617757797241 - rollout_corr/rollout_is_seq_min:0.9970559477806091 - rollout_corr/rollout_is_seq_max_deviation:0.005961775779724121 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4541357709094882) - rollout_corr/training_log_ppl:np.float64(0.3529185401275754) - rollout_corr/kl:np.float64(0.0004487401768882293) - rollout_corr/k3_kl:np.float64(0.00045227643718703803) - rollout_corr/rollout_ppl:np.float64(1.4534654365852475) - rollout_corr/rollout_log_ppl:np.float64(0.3524697999237105) - rollout_corr/log_ppl_diff:np.float64(0.00044874020386487246) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009008654160425067) - rollout_corr/log_ppl_diff_max:np.float64(0.004554241895675659) - rollout_corr/log_ppl_diff_min:np.float64(-0.005111396312713623) - rollout_corr/ppl_ratio:np.float64(1.000449490500614) - rollout_corr/chi2_token:np.float64(0.0009134733118116856) - rollout_corr/chi2_seq:np.float64(13.181665818439797) - actor/pg_loss:np.float64(5.611649248749018e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.645304444598878e-05) - actor/ppo_kl:np.float64(5.086660891162609e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08397138444706798) - perf/mfu/actor:np.float64(0.1864176210922637) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.92216110229492) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02002083510160446 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02002083510160446 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1703.328125 - response_length/max:8192.0 - response_length/min:198.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1703.328125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:198.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.438022971153259e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.710974918678403) - timing_s/agent_loop/generate_sequences/max:np.float64(59.87396731041372) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.808275037481508) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.87396731041372) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.76294595561922 - timing_s/reward:0.1163025051355362 - timing_s/old_log_prob:4.507007392123342 - timing_s/adv:0.12916515208780766 - timing_s/update_actor:18.44666711241007 - timing_s/step:90.04977263323963 - timing_s/stop_profile:0.00011244788765907288 - timing_per_token_ms/gen:0.15310776227518558 - timing_per_token_ms/update_actor:0.03633493888405005 - timing_per_token_ms/adv:0.00025442037190025223 - perf/total_num_tokens:507684 - perf/time_per_step:90.04977263323963 - perf/throughput:704.7269320541865 (TaskRunner pid=2281424) Training Progress: 66%|██████▌ | 66/100 [1:21:20<50:52, 89.79s/it] (TaskRunner pid=2281424) step:67 - global_seqlen/min:29846 - global_seqlen/max:66602 - global_seqlen/minmax_diff:36756 - global_seqlen/balanced_min:48064 - global_seqlen/balanced_max:48303 - global_seqlen/mean:48233.875 - actor/entropy:0.4240332841873169 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18446603417396545 - training/rollout_probs_diff_mean:0.0033334451727569103 - training/rollout_probs_diff_std:0.00801022257655859 - training/rollout_actor_probs_pearson_corr:0.9995075464248657 - rollout_corr/rollout_is_mean:1.000016450881958 - rollout_corr/rollout_is_ratio_fraction_high:3.1456927445105975e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.291385489021195e-06 - rollout_corr/rollout_is_max:2.192669630050659 - rollout_corr/rollout_is_min:0.3415142297744751 - rollout_corr/rollout_is_std:0.028202205896377563 - rollout_corr/rollout_is_eff_sample_size:0.9992053867417628 - rollout_corr/rollout_is_seq_mean:0.9999523758888245 - rollout_corr/rollout_is_seq_std:0.0010149587178602815 - rollout_corr/rollout_is_seq_max:1.0026092529296875 - rollout_corr/rollout_is_seq_min:0.9962970018386841 - rollout_corr/rollout_is_seq_max_deviation:0.003702998161315918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.456437811255455) - rollout_corr/training_log_ppl:np.float64(0.3491874962928705) - rollout_corr/kl:np.float64(0.000549241650548149) - rollout_corr/k3_kl:np.float64(0.0004329512739786878) - rollout_corr/rollout_ppl:np.float64(1.4555674134753644) - rollout_corr/rollout_log_ppl:np.float64(0.3486382510745898) - rollout_corr/log_ppl_diff:np.float64(0.0005492452182807028) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009298735531046987) - rollout_corr/log_ppl_diff_max:np.float64(0.005370736122131348) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034429728984832764) - rollout_corr/ppl_ratio:np.float64(1.0005500647239387) - rollout_corr/chi2_token:np.float64(0.0006130961701273918) - rollout_corr/chi2_seq:np.float64(3.0569681401830167) - actor/pg_loss:np.float64(-7.326365448534489e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.657254923287837e-05) - actor/ppo_kl:np.float64(0.00010237288070413797) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0874917283654213) - perf/mfu/actor:np.float64(0.15933752928759692) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.65837478637695) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010827867314219475 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010827867314219475 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1241.77734375 - response_length/max:6316.0 - response_length/min:226.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1241.77734375 - response_length_non_aborted/max:6316.0 - response_length_non_aborted/min:226.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00024794600903987885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7115938980132341) - timing_s/agent_loop/generate_sequences/max:np.float64(44.07025810703635) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.514710396375449) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.07025810703635) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:6316 - timing_s/gen:51.214709755033255 - timing_s/reward:0.09305312111973763 - timing_s/old_log_prob:4.1541140507906675 - timing_s/adv:0.1196837779134512 - timing_s/update_actor:16.18540153838694 - timing_s/step:71.87573814950883 - timing_s/stop_profile:7.292069494724274e-05 - timing_per_token_ms/gen:0.1611057416915436 - timing_per_token_ms/update_actor:0.041945109993720545 - timing_per_token_ms/adv:0.0003101652570767204 - perf/total_num_tokens:385871 - perf/time_per_step:71.87573814950883 - perf/throughput:671.0731081421194 (TaskRunner pid=2281424) Training Progress: 67%|██████▋ | 67/100 [1:22:33<46:30, 84.55s/it] (TaskRunner pid=2281424) step:68 - global_seqlen/min:29366 - global_seqlen/max:71127 - global_seqlen/minmax_diff:41761 - global_seqlen/balanced_min:45241 - global_seqlen/balanced_max:45620 - global_seqlen/mean:45320.875 - actor/entropy:0.3679567873477936 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846073567867279 - training/rollout_probs_diff_mean:0.003109095385298133 - training/rollout_probs_diff_std:0.007897583767771721 - training/rollout_actor_probs_pearson_corr:0.9995740056037903 - rollout_corr/rollout_is_mean:0.9999691247940063 - rollout_corr/rollout_is_ratio_fraction_high:3.393983888599905e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.393983888599905e-06 - rollout_corr/rollout_is_max:3.361013650894165 - rollout_corr/rollout_is_min:0.496554970741272 - rollout_corr/rollout_is_std:0.02703685127198696 - rollout_corr/rollout_is_eff_sample_size:0.9992695425945294 - rollout_corr/rollout_is_seq_mean:1.0000405311584473 - rollout_corr/rollout_is_seq_std:0.001029360806569457 - rollout_corr/rollout_is_seq_max:1.0070128440856934 - rollout_corr/rollout_is_seq_min:0.9971495866775513 - rollout_corr/rollout_is_seq_max_deviation:0.007012844085693359 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3687643725425005) - rollout_corr/training_log_ppl:np.float64(0.3007245181070175) - rollout_corr/kl:np.float64(0.0003466152552178414) - rollout_corr/k3_kl:np.float64(0.0004064006222961325) - rollout_corr/rollout_ppl:np.float64(1.368270150385797) - rollout_corr/rollout_log_ppl:np.float64(0.3003779045538977) - rollout_corr/log_ppl_diff:np.float64(0.00034661355311982334) - rollout_corr/log_ppl_abs_diff:np.float64(0.000815852457890287) - rollout_corr/log_ppl_diff_max:np.float64(0.0050161778926849365) - rollout_corr/log_ppl_diff_min:np.float64(-0.00573354959487915) - rollout_corr/ppl_ratio:np.float64(1.0003472932148725) - rollout_corr/chi2_token:np.float64(0.0009375796653330326) - rollout_corr/chi2_seq:np.float64(0.4990305930841714) - actor/pg_loss:np.float64(4.833098500967026e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.405877534618412e-05) - actor/ppo_kl:np.float64(4.350960717935948e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09933188185095787) - perf/mfu/actor:np.float64(0.15303388314863475) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.6871109008789) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008664840832352638 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008664840832352638 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1150.93359375 - response_length/max:6380.0 - response_length/min:199.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1150.93359375 - response_length_non_aborted/max:6380.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.791895091533661e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0759412702172995) - timing_s/agent_loop/generate_sequences/max:np.float64(44.35951707884669) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.321704763286107) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.35951707884669) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:6380 - timing_s/gen:51.27926462888718 - timing_s/reward:0.09004293940961361 - timing_s/old_log_prob:3.896944200620055 - timing_s/adv:0.12336954846978188 - timing_s/update_actor:15.747877143323421 - timing_s/step:71.23202810995281 - timing_s/stop_profile:0.00013591721653938293 - timing_per_token_ms/gen:0.1740409946710625 - timing_per_token_ms/update_actor:0.04343439183191913 - timing_per_token_ms/adv:0.0003402668981727015 - perf/total_num_tokens:362567 - perf/time_per_step:71.23202810995281 - perf/throughput:636.2429401847621 (TaskRunner pid=2281424) Training Progress: 68%|██████▊ | 68/100 [1:23:44<42:58, 80.57s/it] (TaskRunner pid=2281424) step:69 - global_seqlen/min:35402 - global_seqlen/max:71677 - global_seqlen/minmax_diff:36275 - global_seqlen/balanced_min:51417 - global_seqlen/balanced_max:51537 - global_seqlen/mean:51488.375 - actor/entropy:0.41125714778900146 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462836742401123 - training/rollout_probs_diff_mean:0.0032182156573981047 - training/rollout_probs_diff_std:0.007813959382474422 - training/rollout_actor_probs_pearson_corr:0.9995726346969604 - rollout_corr/rollout_is_mean:0.9999262094497681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.918830205089762e-06 - rollout_corr/rollout_is_max:1.644830346107483 - rollout_corr/rollout_is_min:0.48185646533966064 - rollout_corr/rollout_is_std:0.027313215658068657 - rollout_corr/rollout_is_eff_sample_size:0.9992545443846275 - rollout_corr/rollout_is_seq_mean:0.9999979734420776 - rollout_corr/rollout_is_seq_std:0.0008565172320231795 - rollout_corr/rollout_is_seq_max:1.0034431219100952 - rollout_corr/rollout_is_seq_min:0.9972632527351379 - rollout_corr/rollout_is_seq_max_deviation:0.003443121910095215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3975837221369147) - rollout_corr/training_log_ppl:np.float64(0.3145434020116227) - rollout_corr/kl:np.float64(0.0003842490903210871) - rollout_corr/k3_kl:np.float64(0.000364644451536833) - rollout_corr/rollout_ppl:np.float64(1.3969996976666152) - rollout_corr/rollout_log_ppl:np.float64(0.31415915075922385) - rollout_corr/log_ppl_diff:np.float64(0.0003842512523988262) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007727831398369744) - rollout_corr/log_ppl_diff_max:np.float64(0.0032014399766921997) - rollout_corr/log_ppl_diff_min:np.float64(-0.002889871597290039) - rollout_corr/ppl_ratio:np.float64(1.0003847263287753) - rollout_corr/chi2_token:np.float64(0.0007037864997982979) - rollout_corr/chi2_seq:np.float64(0.6845616844948381) - actor/pg_loss:np.float64(-3.3041927963495255e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.254580268385325e-05) - actor/ppo_kl:np.float64(3.3652320112764755e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08329814206808805) - perf/mfu/actor:np.float64(0.166977802348238) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.75886535644531) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006434926297515631 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006434926297515631 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1338.29296875 - response_length/max:5606.0 - response_length/min:269.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1338.29296875 - response_length_non_aborted/max:5606.0 - response_length_non_aborted/min:269.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010468810796737671 - timing_s/agent_loop/generate_sequences/min:np.float64(3.5091678723692894) - timing_s/agent_loop/generate_sequences/max:np.float64(41.96201452054083) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.428402047728014) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(41.96201452054083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:5606 - timing_s/gen:48.85266746021807 - timing_s/reward:0.09144083969295025 - timing_s/old_log_prob:3.999005449935794 - timing_s/adv:0.11515455320477486 - timing_s/update_actor:16.556827584281564 - timing_s/step:69.7196654882282 - timing_s/stop_profile:3.0202791094779968e-05 - timing_per_token_ms/gen:0.14259264355600526 - timing_per_token_ms/update_actor:0.04019554798602977 - timing_per_token_ms/adv:0.00027956444829724876 - perf/total_num_tokens:411907 - perf/time_per_step:69.7196654882282 - perf/throughput:738.5057664783768 (TaskRunner pid=2281424) Training Progress: 69%|██████▉ | 69/100 [1:24:54<39:56, 77.32s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the particle's charge, we can use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Calculate $ r^2 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) r^2 = (0.50)^2 = 0.25 \, \mathrm{m^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Plug into the formula (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 4: Choose the closest answer (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option to 55.6 pC is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) **B: 56 pC** (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula for the electric field due to a point charge, $ E = k \frac{q}{r^2} $, and solve for $ q $. We calculate $ r^2 = 0.25 \, \mathrm{m^2} $, plug into the formula, and find $ q \approx 5.56 \times 10^{-11} \, \mathrm{C} $, which is approximately 55.6 pC. The closest answer is B. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2281424) step:70 - global_seqlen/min:28634 - global_seqlen/max:74589 - global_seqlen/minmax_diff:45955 - global_seqlen/balanced_min:51601 - global_seqlen/balanced_max:52935 - global_seqlen/mean:51969.25 - actor/entropy:0.463895708322525 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6431382894515991 - training/rollout_probs_diff_mean:0.0034441002644598484 - training/rollout_probs_diff_std:0.008035892620682716 - training/rollout_actor_probs_pearson_corr:0.9996177554130554 - rollout_corr/rollout_is_mean:0.9999815225601196 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8799022402381524e-06 - rollout_corr/rollout_is_max:1.7186620235443115 - rollout_corr/rollout_is_min:0.18310591578483582 - rollout_corr/rollout_is_std:0.028500739485025406 - rollout_corr/rollout_is_eff_sample_size:0.9991883671839248 - rollout_corr/rollout_is_seq_mean:1.0000474452972412 - rollout_corr/rollout_is_seq_std:0.0009564801002852619 - rollout_corr/rollout_is_seq_max:1.0044219493865967 - rollout_corr/rollout_is_seq_min:0.9975866675376892 - rollout_corr/rollout_is_seq_max_deviation:0.00442194938659668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4960705381818116) - rollout_corr/training_log_ppl:np.float64(0.38029307904071175) - rollout_corr/kl:np.float64(0.00034536135440355054) - rollout_corr/k3_kl:np.float64(0.0003967561805495734) - rollout_corr/rollout_ppl:np.float64(1.4955097283236682) - rollout_corr/rollout_log_ppl:np.float64(0.3799477120919619) - rollout_corr/log_ppl_diff:np.float64(0.00034536694874987006) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008130436181090772) - rollout_corr/log_ppl_diff_max:np.float64(0.0031409859657287598) - rollout_corr/log_ppl_diff_min:np.float64(-0.002258613705635071) - rollout_corr/ppl_ratio:np.float64(1.0003458971623331) - rollout_corr/chi2_token:np.float64(0.0009016424883157015) - rollout_corr/chi2_seq:np.float64(2.9101600360590965) - actor/pg_loss:np.float64(-6.507872603833675e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.506012740852384e-05) - actor/ppo_kl:np.float64(7.824350067409114e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11454777047038078) - perf/mfu/actor:np.float64(0.16590592651685154) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.71920394897461) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.68125) - val-aux/sciknoweval/reward/std@16:np.float64(0.2137127463842002) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.769025) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1809424350916951) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5937875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1784672156379507) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6828625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21320174521790633) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8434625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1347694924147861) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5284375) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13266359016202112) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.711225) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17773838845597045) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8962875) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0854184909297855) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47755000000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10070722739093489) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7298625000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14406533830862692) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9257875) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.045417892588033096) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4352875) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07488497149607146) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7414875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11564756114569612) - val-aux/sciknoweval/score/mean@16:np.float64(0.68125) - val-aux/sciknoweval/score/std@16:np.float64(0.2137127463842002) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.769025) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1809424350916951) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5937875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1784672156379507) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6828625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21320174521790633) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8434625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1347694924147861) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5284375) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13266359016202112) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.711225) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17773838845597045) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8962875) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0854184909297855) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47755000000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10070722739093489) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7298625000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14406533830862692) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9257875) - val-aux/sciknoweval/score/best@16/std:np.float64(0.045417892588033096) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4352875) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07488497149607146) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7414875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11564756114569612) - val-core/sciknoweval/acc/mean@16:np.float64(0.68125) - val-aux/sciknoweval/acc/std@16:np.float64(0.2137127463842002) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.769025) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1809424350916951) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5937875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1784672156379507) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6828625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21320174521790633) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8434625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1347694924147861) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5284375) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13266359016202112) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.711225) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17773838845597045) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8962875) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0854184909297855) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47755000000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10070722739093489) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7298625000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14406533830862692) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9257875) - val-core/sciknoweval/acc/best@16/std:np.float64(0.045417892588033096) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4352875) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07488497149607146) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7414875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11564756114569612) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.12121925691237849) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9806625) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05945683002695111) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.89495) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.1434002448368515) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.93955) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11972140703255145) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9960875000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.01660654402930483) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8371375000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.15424155659422464) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9604250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.09268990099722656) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.999625) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0033009411325088988) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7685375000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.14442824247076974) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9791624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.061597817355926635) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7031375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.11075323791783456) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9912124999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03468685742406086) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004505247343331575 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.004505247343331575 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:1356.3828125 - response_length/max:7204.0 - response_length/min:235.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1356.3828125 - response_length_non_aborted/max:7204.0 - response_length_non_aborted/min:235.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.293583333492279e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.054271874949336) - timing_s/agent_loop/generate_sequences/max:np.float64(50.83749855682254) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.613497315665882) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.83749855682254) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:225 - timing_s/agent_loop/slowest/response_length:7204 - timing_s/gen:57.93693597614765 - timing_s/reward:0.1021033264696598 - timing_s/old_log_prob:4.041707821190357 - timing_s/adv:0.12145847268402576 - timing_s/update_actor:16.889128616079688 - timing_s/step:79.18048135004938 - timing_s/testing:186.86356358602643 - timing_s/save_checkpoint:13.130839332938194 - timing_s/stop_profile:0.00013073720037937164 - timing_per_token_ms/gen:0.16685271596717963 - timing_per_token_ms/update_actor:0.04062288905477683 - timing_per_token_ms/adv:0.00029214023841989676 - perf/total_num_tokens:415754 - perf/time_per_step:79.18048135004938 - perf/throughput:656.3391521989982 (TaskRunner pid=2281424) Training Progress: 70%|███████ | 70/100 [1:29:33<1:08:57, 137.90s/it] (TaskRunner pid=2281424) step:71 - global_seqlen/min:28875 - global_seqlen/max:77315 - global_seqlen/minmax_diff:48440 - global_seqlen/balanced_min:54005 - global_seqlen/balanced_max:54274 - global_seqlen/mean:54174.25 - actor/entropy:0.420095831155777 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42047184705734253 - training/rollout_probs_diff_mean:0.00322735495865345 - training/rollout_probs_diff_std:0.007836757227778435 - training/rollout_actor_probs_pearson_corr:0.999700129032135 - rollout_corr/rollout_is_mean:0.9999887943267822 - rollout_corr/rollout_is_ratio_fraction_high:2.7597932330536423e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.519586466107285e-06 - rollout_corr/rollout_is_max:2.1141860485076904 - rollout_corr/rollout_is_min:0.4562623202800751 - rollout_corr/rollout_is_std:0.02742645889520645 - rollout_corr/rollout_is_eff_sample_size:0.9992484738086154 - rollout_corr/rollout_is_seq_mean:0.9999967813491821 - rollout_corr/rollout_is_seq_std:0.0009134845458902419 - rollout_corr/rollout_is_seq_max:1.0040614604949951 - rollout_corr/rollout_is_seq_min:0.997020423412323 - rollout_corr/rollout_is_seq_max_deviation:0.004061460494995117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4078892222605646) - rollout_corr/training_log_ppl:np.float64(0.3253569998778403) - rollout_corr/kl:np.float64(0.00031806869935557813) - rollout_corr/k3_kl:np.float64(0.00037636621712522356) - rollout_corr/rollout_ppl:np.float64(1.4074036730453372) - rollout_corr/rollout_log_ppl:np.float64(0.3250389306922443) - rollout_corr/log_ppl_diff:np.float64(0.0003180691855959594) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007455509621649981) - rollout_corr/log_ppl_diff_max:np.float64(0.004639908671379089) - rollout_corr/log_ppl_diff_min:np.float64(-0.003470480442047119) - rollout_corr/ppl_ratio:np.float64(1.0003185661043972) - rollout_corr/chi2_token:np.float64(0.0008722511120140553) - rollout_corr/chi2_seq:np.float64(0.7835535886697471) - actor/pg_loss:np.float64(8.003273978829384e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.337292018723019e-05) - actor/ppo_kl:np.float64(-4.260811079248583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10855482891201973) - perf/mfu/actor:np.float64(0.16889551806942366) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.48225784301758) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017720285803079605 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017720285803079605 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1415.4140625 - response_length/max:7054.0 - response_length/min:226.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1415.4140625 - response_length_non_aborted/max:7054.0 - response_length_non_aborted/min:226.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001040436327457428 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8338771276175976) - timing_s/agent_loop/generate_sequences/max:np.float64(49.44516071304679) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.63207975136902) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.44516071304679) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:7054 - timing_s/gen:55.94673459418118 - timing_s/reward:0.09744138084352016 - timing_s/old_log_prob:4.294410843402147 - timing_s/adv:0.1294803023338318 - timing_s/update_actor:17.14490484073758 - timing_s/step:77.71968797035515 - timing_s/stop_profile:0.00013727881014347076 - timing_per_token_ms/gen:0.15440141354998035 - timing_per_token_ms/update_actor:0.03955962666935302 - timing_per_token_ms/adv:0.00029875887145145477 - perf/total_num_tokens:433394 - perf/time_per_step:77.71968797035515 - perf/throughput:697.0466739478398 (TaskRunner pid=2281424) Training Progress: 71%|███████ | 71/100 [1:30:51<57:56, 119.87s/it] (TaskRunner pid=2281424) step:72 - global_seqlen/min:33672 - global_seqlen/max:79500 - global_seqlen/minmax_diff:45828 - global_seqlen/balanced_min:55942 - global_seqlen/balanced_max:56271 - global_seqlen/mean:56194.875 - actor/entropy:0.40107372403144836 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7170437574386597 - training/rollout_probs_diff_mean:0.003104705596342683 - training/rollout_probs_diff_std:0.007854383438825607 - training/rollout_actor_probs_pearson_corr:0.9995812177658081 - rollout_corr/rollout_is_mean:0.9999414682388306 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.225766017247224e-06 - rollout_corr/rollout_is_max:1.9946351051330566 - rollout_corr/rollout_is_min:0.23694215714931488 - rollout_corr/rollout_is_std:0.026999348774552345 - rollout_corr/rollout_is_eff_sample_size:0.9992714471613361 - rollout_corr/rollout_is_seq_mean:0.9999212026596069 - rollout_corr/rollout_is_seq_std:0.0008411617600359023 - rollout_corr/rollout_is_seq_max:1.0026910305023193 - rollout_corr/rollout_is_seq_min:0.9967179298400879 - rollout_corr/rollout_is_seq_max_deviation:0.0032820701599121094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.386727336794138) - rollout_corr/training_log_ppl:np.float64(0.31231660785852) - rollout_corr/kl:np.float64(0.00041095774418131725) - rollout_corr/k3_kl:np.float64(0.0003522394192145839) - rollout_corr/rollout_ppl:np.float64(1.3861299636773765) - rollout_corr/rollout_log_ppl:np.float64(0.3119056501018349) - rollout_corr/log_ppl_diff:np.float64(0.00041095775668509305) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007225902227219194) - rollout_corr/log_ppl_diff_max:np.float64(0.0031125247478485107) - rollout_corr/log_ppl_diff_min:np.float64(-0.00223328173160553) - rollout_corr/ppl_ratio:np.float64(1.00041141663678) - rollout_corr/chi2_token:np.float64(0.0005854771006852388) - rollout_corr/chi2_seq:np.float64(3.2485340386629105) - actor/pg_loss:np.float64(1.788930967450142e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.609930801459996e-05) - actor/ppo_kl:np.float64(-7.207803651265721e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07063721679151058) - perf/mfu/actor:np.float64(0.1745573387991699) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.36584854125977) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016829580068588257 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016829580068588257 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1494.99609375 - response_length/max:7029.0 - response_length/min:275.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1494.99609375 - response_length_non_aborted/max:7029.0 - response_length_non_aborted/min:275.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001059994101524353 - timing_s/agent_loop/generate_sequences/min:np.float64(3.654697211459279) - timing_s/agent_loop/generate_sequences/max:np.float64(50.904972886666656) - timing_s/agent_loop/generate_sequences/mean:np.float64(15.098586545231228) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.904972886666656) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:7029 - timing_s/gen:58.266811951994896 - timing_s/reward:0.106016606092453 - timing_s/old_log_prob:4.172686280682683 - timing_s/adv:0.13374079205095768 - timing_s/update_actor:17.26791531406343 - timing_s/step:80.03169925138354 - timing_s/stop_profile:3.802776336669922e-05 - timing_per_token_ms/gen:0.1522443671518657 - timing_per_token_ms/update_actor:0.038410787714323213 - timing_per_token_ms/adv:0.000297493303550719 - perf/total_num_tokens:449559 - perf/time_per_step:80.03169925138354 - perf/throughput:702.1577140763825 (TaskRunner pid=2281424) Training Progress: 72%|███████▏ | 72/100 [1:32:11<50:21, 107.92s/it] (TaskRunner pid=2281424) step:73 - global_seqlen/min:50710 - global_seqlen/max:100431 - global_seqlen/minmax_diff:49721 - global_seqlen/balanced_min:71047 - global_seqlen/balanced_max:71387 - global_seqlen/mean:71271.125 - actor/entropy:0.4723150432109833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466041564941406 - training/rollout_probs_diff_mean:0.0032528568990528584 - training/rollout_probs_diff_std:0.007730447221547365 - training/rollout_actor_probs_pearson_corr:0.9995112419128418 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:4.0056320358417e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.0084480537625495e-06 - rollout_corr/rollout_is_max:2.142465353012085 - rollout_corr/rollout_is_min:0.40851542353630066 - rollout_corr/rollout_is_std:0.028115419670939445 - rollout_corr/rollout_is_eff_sample_size:0.9992101475610164 - rollout_corr/rollout_is_seq_mean:0.9999721646308899 - rollout_corr/rollout_is_seq_std:0.0010153069160878658 - rollout_corr/rollout_is_seq_max:1.0036121606826782 - rollout_corr/rollout_is_seq_min:0.9954786896705627 - rollout_corr/rollout_is_seq_max_deviation:0.004521310329437256 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5221755541861057) - rollout_corr/training_log_ppl:np.float64(0.39488993302802555) - rollout_corr/kl:np.float64(0.0004714057424735074) - rollout_corr/k3_kl:np.float64(0.0004070275241190302) - rollout_corr/rollout_ppl:np.float64(1.5213952930644155) - rollout_corr/rollout_log_ppl:np.float64(0.39441852559684776) - rollout_corr/log_ppl_diff:np.float64(0.00047140743117779493) - rollout_corr/log_ppl_abs_diff:np.float64(0.000822892296127975) - rollout_corr/log_ppl_diff_max:np.float64(0.0034640729427337646) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030289292335510254) - rollout_corr/ppl_ratio:np.float64(1.0004719870630652) - rollout_corr/chi2_token:np.float64(0.0006970379035919905) - rollout_corr/chi2_seq:np.float64(1.1741749560460448) - actor/pg_loss:np.float64(8.69284849613905e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010152803969276647) - actor/ppo_kl:np.float64(4.153459515521263e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0953609012067318) - perf/mfu/actor:np.float64(0.19884977160610307) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.27853107452393) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005314472131431103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005314472131431103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1950.37890625 - response_length/max:7405.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1950.37890625 - response_length_non_aborted/max:7405.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.626864731311798e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4811857379972935) - timing_s/agent_loop/generate_sequences/max:np.float64(56.79083485901356) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.615477024730353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(56.79083485901356) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:7405 - timing_s/gen:64.19048333540559 - timing_s/reward:0.1299944929778576 - timing_s/old_log_prob:4.7613958436995745 - timing_s/adv:0.13127423450350761 - timing_s/update_actor:19.186476746574044 - timing_s/step:88.48862772993743 - timing_s/stop_profile:0.00012197904288768768 - timing_per_token_ms/gen:0.12856172445539546 - timing_per_token_ms/update_actor:0.03365050843973286 - timing_per_token_ms/adv:0.00023023741119476438 - perf/total_num_tokens:570169 - perf/time_per_step:88.48862772993743 - perf/throughput:805.426943872558 (TaskRunner pid=2281424) Training Progress: 73%|███████▎ | 73/100 [1:33:39<45:56, 102.11s/it] (TaskRunner pid=2281424) step:74 - global_seqlen/min:37867 - global_seqlen/max:74272 - global_seqlen/minmax_diff:36405 - global_seqlen/balanced_min:53139 - global_seqlen/balanced_max:53734 - global_seqlen/mean:53577.125 - actor/entropy:0.44166502356529236 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30947351455688477 - training/rollout_probs_diff_mean:0.0033270702697336674 - training/rollout_probs_diff_std:0.007980842143297195 - training/rollout_actor_probs_pearson_corr:0.9996140599250793 - rollout_corr/rollout_is_mean:0.9999847412109375 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7457967917143833e-06 - rollout_corr/rollout_is_max:1.913217306137085 - rollout_corr/rollout_is_min:0.3732314705848694 - rollout_corr/rollout_is_std:0.02833504043519497 - rollout_corr/rollout_is_eff_sample_size:0.9991978885434327 - rollout_corr/rollout_is_seq_mean:1.000065565109253 - rollout_corr/rollout_is_seq_std:0.0012911416124552488 - rollout_corr/rollout_is_seq_max:1.006355881690979 - rollout_corr/rollout_is_seq_min:0.9940794706344604 - rollout_corr/rollout_is_seq_max_deviation:0.006355881690979004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.46239750739187) - rollout_corr/training_log_ppl:np.float64(0.35442679983680137) - rollout_corr/kl:np.float64(0.0003367374671343626) - rollout_corr/k3_kl:np.float64(0.00044517193822457557) - rollout_corr/rollout_ppl:np.float64(1.4618395678699017) - rollout_corr/rollout_log_ppl:np.float64(0.3540900625521317) - rollout_corr/log_ppl_diff:np.float64(0.0003367372846696526) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008827941201161593) - rollout_corr/log_ppl_diff_max:np.float64(0.006804674863815308) - rollout_corr/log_ppl_diff_min:np.float64(-0.005605608224868774) - rollout_corr/ppl_ratio:np.float64(1.0003375951200724) - rollout_corr/chi2_token:np.float64(0.0011235442943871021) - rollout_corr/chi2_seq:np.float64(0.879344102460891) - actor/pg_loss:np.float64(2.2439053282141685e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.060110855012681e-05) - actor/ppo_kl:np.float64(-3.356134554266532e-05) - actor/pg_clipfrac_lower:np.float64(1.6035509133871528e-06) - actor/grad_norm:np.float64(0.09511143900454044) - perf/mfu/actor:np.float64(0.16871793536919505) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.22051620483398) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0102184833958745 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0102184833958745 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1422.62890625 - response_length/max:7451.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1422.62890625 - response_length_non_aborted/max:7451.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001026596873998642 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8556665536016226) - timing_s/agent_loop/generate_sequences/max:np.float64(52.814375065267086) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.79034505609161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(52.814375065267086) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:7451 - timing_s/gen:60.13706563971937 - timing_s/reward:0.1039069052785635 - timing_s/old_log_prob:4.208096336573362 - timing_s/adv:0.12475542351603508 - timing_s/update_actor:17.123554781079292 - timing_s/step:81.78595715388656 - timing_s/stop_profile:0.00013553164899349213 - timing_per_token_ms/gen:0.16512416669106592 - timing_per_token_ms/update_actor:0.03995071306336261 - timing_per_token_ms/adv:0.00029106503828834387 - perf/total_num_tokens:428617 - perf/time_per_step:81.78595715388656 - perf/throughput:655.0895393837665 (TaskRunner pid=2281424) Training Progress: 74%|███████▍ | 74/100 [1:35:01<41:36, 96.02s/it] (TaskRunner pid=2281424) step:75 - global_seqlen/min:29725 - global_seqlen/max:63241 - global_seqlen/minmax_diff:33516 - global_seqlen/balanced_min:43431 - global_seqlen/balanced_max:43829 - global_seqlen/mean:43649.625 - actor/entropy:0.40172678232192993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4140506982803345 - training/rollout_probs_diff_mean:0.0032765574287623167 - training/rollout_probs_diff_std:0.008161391131579876 - training/rollout_actor_probs_pearson_corr:0.9995269775390625 - rollout_corr/rollout_is_mean:1.0000414848327637 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0609216587909032e-05 - rollout_corr/rollout_is_max:1.553276777267456 - rollout_corr/rollout_is_min:0.2633557915687561 - rollout_corr/rollout_is_std:0.027534905821084976 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:1.0000547170639038 - rollout_corr/rollout_is_seq_std:0.0010406997753307223 - rollout_corr/rollout_is_seq_max:1.0065134763717651 - rollout_corr/rollout_is_seq_min:0.9971379041671753 - rollout_corr/rollout_is_seq_max_deviation:0.006513476371765137 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.368973680306226) - rollout_corr/training_log_ppl:np.float64(0.29217190519557334) - rollout_corr/kl:np.float64(0.0003027983755394814) - rollout_corr/k3_kl:np.float64(0.0003719866110429848) - rollout_corr/rollout_ppl:np.float64(1.3685665079392493) - rollout_corr/rollout_log_ppl:np.float64(0.29186910606222227) - rollout_corr/log_ppl_diff:np.float64(0.00030279913335107267) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008351847936864942) - rollout_corr/log_ppl_diff_max:np.float64(0.00448000431060791) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031926631927490234) - rollout_corr/ppl_ratio:np.float64(1.000303411623463) - rollout_corr/chi2_token:np.float64(0.0008776339236646891) - rollout_corr/chi2_seq:np.float64(2.008884164271876) - actor/pg_loss:np.float64(2.7343048714101315e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.3585604796826374e-05) - actor/ppo_kl:np.float64(-8.158876836361628e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.034894744865596294) - perf/mfu/actor:np.float64(0.14880178274967848) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.15599822998047) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.85546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.85546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015577865997329354 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015577865997329354 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1104.58203125 - response_length/max:7265.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1104.58203125 - response_length_non_aborted/max:7265.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001430269330739975 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6985986921936274) - timing_s/agent_loop/generate_sequences/max:np.float64(49.74266072921455) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.12252202147647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.74266072921455) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:7265 - timing_s/gen:56.838472582399845 - timing_s/reward:0.08878145180642605 - timing_s/old_log_prob:3.778964426368475 - timing_s/adv:0.15880484133958817 - timing_s/update_actor:15.498764231801033 - timing_s/step:76.44660468026996 - timing_s/stop_profile:0.00011175312101840973 - timing_per_token_ms/gen:0.20100388856927587 - timing_per_token_ms/update_actor:0.04438401312669076 - timing_per_token_ms/adv:0.0004547714938547243 - perf/total_num_tokens:349197 - perf/time_per_step:76.44660468026996 - perf/throughput:570.9818661346708 (TaskRunner pid=2281424) Training Progress: 75%|███████▌ | 75/100 [1:36:18<37:34, 90.17s/it] (TaskRunner pid=2281424) step:76 - global_seqlen/min:42462 - global_seqlen/max:96092 - global_seqlen/minmax_diff:53630 - global_seqlen/balanced_min:72056 - global_seqlen/balanced_max:72271 - global_seqlen/mean:72168.875 - actor/entropy:0.4893510341644287 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2547096014022827 - training/rollout_probs_diff_mean:0.003322131233289838 - training/rollout_probs_diff_std:0.0077032484114170074 - training/rollout_actor_probs_pearson_corr:0.9996566772460938 - rollout_corr/rollout_is_mean:1.0001403093338013 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9754140794248087e-06 - rollout_corr/rollout_is_max:1.7543652057647705 - rollout_corr/rollout_is_min:0.48901525139808655 - rollout_corr/rollout_is_std:0.028295045718550682 - rollout_corr/rollout_is_eff_sample_size:0.9992002689116629 - rollout_corr/rollout_is_seq_mean:1.0001996755599976 - rollout_corr/rollout_is_seq_std:0.001209214678965509 - rollout_corr/rollout_is_seq_max:1.009247064590454 - rollout_corr/rollout_is_seq_min:0.9936361908912659 - rollout_corr/rollout_is_seq_max_deviation:0.009247064590454102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5176880699582398) - rollout_corr/training_log_ppl:np.float64(0.3912102092872374) - rollout_corr/kl:np.float64(0.0003245452827291828) - rollout_corr/k3_kl:np.float64(0.0004195083137688016) - rollout_corr/rollout_ppl:np.float64(1.5171602945774794) - rollout_corr/rollout_log_ppl:np.float64(0.39088566205464303) - rollout_corr/log_ppl_diff:np.float64(0.00032454723259434104) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007810101378709078) - rollout_corr/log_ppl_diff_max:np.float64(0.007137387990951538) - rollout_corr/log_ppl_diff_min:np.float64(-0.008129239082336426) - rollout_corr/ppl_ratio:np.float64(1.000325285596773) - rollout_corr/chi2_token:np.float64(0.0010407264344394207) - rollout_corr/chi2_seq:np.float64(1.3129587173461914) - actor/pg_loss:np.float64(1.1760275810956955e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.053675117025705e-05) - actor/ppo_kl:np.float64(0.00012599972318838581) - actor/pg_clipfrac_lower:np.float64(2.287031520609162e-06) - actor/grad_norm:np.float64(0.09297847654670477) - perf/mfu/actor:np.float64(0.20044092158481716) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.21371841430664) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.027344420552253723 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.027344420552253723 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1977.43359375 - response_length/max:6976.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1977.43359375 - response_length_non_aborted/max:6976.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.447336196899414e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7599249705672264) - timing_s/agent_loop/generate_sequences/max:np.float64(53.188171891495585) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.02186474698101) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(53.188171891495585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:6976 - timing_s/gen:60.21184345707297 - timing_s/reward:0.11902121268212795 - timing_s/old_log_prob:4.799457537010312 - timing_s/adv:0.14908457174897194 - timing_s/update_actor:19.46632076613605 - timing_s/step:84.86317491158843 - timing_s/stop_profile:0.00012650899589061737 - timing_per_token_ms/gen:0.11894331837366728 - timing_per_token_ms/update_actor:0.03371661392486728 - timing_per_token_ms/adv:0.0002582217260366258 - perf/total_num_tokens:577351 - perf/time_per_step:84.86317491158843 - perf/throughput:850.414506353156 (TaskRunner pid=2281424) Training Progress: 76%|███████▌ | 76/100 [1:37:43<35:26, 88.59s/it] (TaskRunner pid=2281424) step:77 - global_seqlen/min:27511 - global_seqlen/max:128154 - global_seqlen/minmax_diff:100643 - global_seqlen/balanced_min:64748 - global_seqlen/balanced_max:65174 - global_seqlen/mean:65054.25 - actor/entropy:0.462461918592453 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6478930115699768 - training/rollout_probs_diff_mean:0.0033272244036197662 - training/rollout_probs_diff_std:0.007839487865567207 - training/rollout_actor_probs_pearson_corr:0.9997078776359558 - rollout_corr/rollout_is_mean:1.000058650970459 - rollout_corr/rollout_is_ratio_fraction_high:2.2016636194166495e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.604990630876273e-06 - rollout_corr/rollout_is_max:4.972353458404541 - rollout_corr/rollout_is_min:0.14135953783988953 - rollout_corr/rollout_is_std:0.02817683294415474 - rollout_corr/rollout_is_eff_sample_size:0.9992068149827754 - rollout_corr/rollout_is_seq_mean:1.0000437498092651 - rollout_corr/rollout_is_seq_std:0.0011115683009847999 - rollout_corr/rollout_is_seq_max:1.0068318843841553 - rollout_corr/rollout_is_seq_min:0.9928021430969238 - rollout_corr/rollout_is_seq_max_deviation:0.007197856903076172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4656865671277046) - rollout_corr/training_log_ppl:np.float64(0.36309026755043305) - rollout_corr/kl:np.float64(0.00044725021728453385) - rollout_corr/k3_kl:np.float64(0.0003817609845384595) - rollout_corr/rollout_ppl:np.float64(1.4650072930380702) - rollout_corr/rollout_log_ppl:np.float64(0.3626430144358892) - rollout_corr/log_ppl_diff:np.float64(0.0004472531145438552) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008061645785346627) - rollout_corr/log_ppl_diff_max:np.float64(0.008196413516998291) - rollout_corr/log_ppl_diff_min:np.float64(-0.0022748708724975586) - rollout_corr/ppl_ratio:np.float64(1.0004479088820517) - rollout_corr/chi2_token:np.float64(0.0006361156702041626) - rollout_corr/chi2_seq:np.float64(3.7252589049749076) - actor/pg_loss:np.float64(4.542234819382429e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.581747235008152e-05) - actor/ppo_kl:np.float64(0.00011224961554034962) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09250594861805439) - perf/mfu/actor:np.float64(0.1876084141306772) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.25350570678711) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009082688018679619 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009082688018679619 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1774.2265625 - response_length/max:8192.0 - response_length/min:214.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1774.2265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:214.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010951049625873566 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6410596389323473) - timing_s/agent_loop/generate_sequences/max:np.float64(66.44215983897448) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.0781091665267) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(66.44215983897448) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:73.64733845368028 - timing_s/reward:0.11625036410987377 - timing_s/old_log_prob:4.594569396227598 - timing_s/adv:0.1430046670138836 - timing_s/update_actor:18.86780060455203 - timing_s/step:97.45536208339036 - timing_s/stop_profile:0.0001203492283821106 - timing_per_token_ms/gen:0.16214666261636954 - timing_per_token_ms/update_actor:0.036253973807537615 - timing_per_token_ms/adv:0.0002747796397120165 - perf/total_num_tokens:520434 - perf/time_per_step:97.45536208339036 - perf/throughput:667.5286881016823 (TaskRunner pid=2281424) Training Progress: 77%|███████▋ | 77/100 [1:39:20<34:59, 91.27s/it] (TaskRunner pid=2281424) step:78 - global_seqlen/min:25325 - global_seqlen/max:84216 - global_seqlen/minmax_diff:58891 - global_seqlen/balanced_min:56720 - global_seqlen/balanced_max:56942 - global_seqlen/mean:56839.875 - actor/entropy:0.44374775886535645 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491605162620544 - training/rollout_probs_diff_mean:0.003467120695859194 - training/rollout_probs_diff_std:0.008032586425542831 - training/rollout_actor_probs_pearson_corr:0.9996151328086853 - rollout_corr/rollout_is_mean:0.9999576210975647 - rollout_corr/rollout_is_ratio_fraction_high:7.75131684349617e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.75131684349617e-06 - rollout_corr/rollout_is_max:4.747284889221191 - rollout_corr/rollout_is_min:0.10899676382541656 - rollout_corr/rollout_is_std:0.02860928140580654 - rollout_corr/rollout_is_eff_sample_size:0.9991820593831796 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.001185454661026597 - rollout_corr/rollout_is_seq_max:1.005151629447937 - rollout_corr/rollout_is_seq_min:0.9958641529083252 - rollout_corr/rollout_is_seq_max_deviation:0.005151629447937012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4343888186849654) - rollout_corr/training_log_ppl:np.float64(0.3459277451329399) - rollout_corr/kl:np.float64(0.0004222604492056803) - rollout_corr/k3_kl:np.float64(0.0003943129925687572) - rollout_corr/rollout_ppl:np.float64(1.4337338921613991) - rollout_corr/rollout_log_ppl:np.float64(0.34550548443803564) - rollout_corr/log_ppl_diff:np.float64(0.0004222606949042529) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008823849784675986) - rollout_corr/log_ppl_diff_max:np.float64(0.004985526204109192) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038795918226242065) - rollout_corr/ppl_ratio:np.float64(1.0004229948390275) - rollout_corr/chi2_token:np.float64(0.0007329215295612812) - rollout_corr/chi2_seq:np.float64(0.7212347930762917) - actor/pg_loss:np.float64(6.53313472867012e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.447403512765959e-05) - actor/ppo_kl:np.float64(2.688655172633503e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08514224551618099) - perf/mfu/actor:np.float64(0.17315237393578076) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.3226547241211) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06087884306907654 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06087884306907654 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1511.83984375 - response_length/max:8192.0 - response_length/min:229.0 - response_length/clip_ratio:0.01953125 - response_length_non_aborted/mean:1511.83984375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:229.0 - response_length_non_aborted/clip_ratio:0.01953125 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010377727448940277 - timing_s/agent_loop/generate_sequences/min:np.float64(3.003926744684577) - timing_s/agent_loop/generate_sequences/max:np.float64(69.25940157100558) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.432622163862106) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.25940157100558) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.2985665909946 - timing_s/reward:0.10811152122914791 - timing_s/old_log_prob:4.204658040776849 - timing_s/adv:0.17462857253849506 - timing_s/update_actor:17.65695309266448 - timing_s/step:98.5296196397394 - timing_s/stop_profile:0.00012289360165596008 - timing_per_token_ms/gen:0.19713812741355238 - timing_per_token_ms/update_actor:0.0388304713299081 - timing_per_token_ms/adv:0.00038403623455033785 - perf/total_num_tokens:454719 - perf/time_per_step:98.5296196397394 - perf/throughput:576.8810963426788 (TaskRunner pid=2281424) Training Progress: 78%|███████▊ | 78/100 [1:40:59<34:16, 93.46s/it] (TaskRunner pid=2281424) step:79 - global_seqlen/min:37448 - global_seqlen/max:108817 - global_seqlen/minmax_diff:71369 - global_seqlen/balanced_min:70970 - global_seqlen/balanced_max:71124 - global_seqlen/mean:71023.25 - actor/entropy:0.4368029832839966 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647865533828735 - training/rollout_probs_diff_mean:0.0031526105012744665 - training/rollout_probs_diff_std:0.007653329521417618 - training/rollout_actor_probs_pearson_corr:0.9996246695518494 - rollout_corr/rollout_is_mean:1.0000195503234863 - rollout_corr/rollout_is_ratio_fraction_high:6.0216539168322925e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.014435944554862e-06 - rollout_corr/rollout_is_max:3.049748420715332 - rollout_corr/rollout_is_min:1.032992599903082e-06 - rollout_corr/rollout_is_std:0.02727172151207924 - rollout_corr/rollout_is_eff_sample_size:0.9992569250228056 - rollout_corr/rollout_is_seq_mean:1.0000171661376953 - rollout_corr/rollout_is_seq_std:0.001091465470381081 - rollout_corr/rollout_is_seq_max:1.005435824394226 - rollout_corr/rollout_is_seq_min:0.9938186407089233 - rollout_corr/rollout_is_seq_max_deviation:0.00618135929107666 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3994069588370621) - rollout_corr/training_log_ppl:np.float64(0.3162277852825355) - rollout_corr/kl:np.float64(0.00035690159927703746) - rollout_corr/k3_kl:np.float64(0.0004051223161241069) - rollout_corr/rollout_ppl:np.float64(1.3988550025969744) - rollout_corr/rollout_log_ppl:np.float64(0.31587088617379777) - rollout_corr/log_ppl_diff:np.float64(0.00035689910873770714) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008177596027962863) - rollout_corr/log_ppl_diff_max:np.float64(0.008748382329940796) - rollout_corr/log_ppl_diff_min:np.float64(-0.00391659140586853) - rollout_corr/ppl_ratio:np.float64(1.0003577324096113) - rollout_corr/chi2_token:np.float64(0.0008362571243196726) - rollout_corr/chi2_seq:np.float64(119.7799857633654) - actor/pg_loss:np.float64(3.9536505937576294e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.417913894234516e-05) - actor/ppo_kl:np.float64(-1.3414459802518763e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.056897282134741545) - perf/mfu/actor:np.float64(0.19757145862440487) - perf/max_memory_allocated_gb:np.float64(133.4042763710022) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.21912050247192) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017608821392059326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017608821392059326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1946.1015625 - response_length/max:8192.0 - response_length/min:209.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:1946.1015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014724023640155792 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6493716575205326) - timing_s/agent_loop/generate_sequences/max:np.float64(73.67174332775176) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.573477208301483) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.67174332775176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:80.96080801077187 - timing_s/reward:0.12271188385784626 - timing_s/old_log_prob:4.715622756630182 - timing_s/adv:0.1407144833356142 - timing_s/update_actor:19.42637729458511 - timing_s/step:105.45251621864736 - timing_s/stop_profile:0.00012031197547912598 - timing_per_token_ms/gen:0.16250598755278356 - timing_per_token_ms/update_actor:0.034190172398800935 - timing_per_token_ms/adv:0.00024765566792496506 - perf/total_num_tokens:568186 - perf/time_per_step:105.45251621864736 - perf/throughput:673.5092963807423 (TaskRunner pid=2281424) Training Progress: 79%|███████▉ | 79/100 [1:42:44<33:58, 97.08s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the charge, we use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Plug in the values (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Compute $ (0.50)^2 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (0.50)^2 = 0.25 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Multiply by 2.0 (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 2.0 \cdot 0.25 = 0.50 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 4: Divide by $ 8.99 \times 10^9 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 5: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 1 \, \mathrm{pC} = 1 \times 10^{-12} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 6: Choose the closest answer (TaskRunner pid=2281424) (TaskRunner pid=2281424) The options are: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - A: 50 pC (TaskRunner pid=2281424) - B: 56 pC (TaskRunner pid=2281424) - C: 60 pC (TaskRunner pid=2281424) - D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest value to 55.6 pC is **56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula $ E = k \frac{q}{r^2} $ to find the charge $ q $. Given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $, we rearrange the formula to solve for $ q $. Calculating the result gives $ q \approx 5.56 \times 10^{-11} \, \mathrm{C} $, which is approximately 55.6 pC. The closest option is B: 56 pC. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2281424) step:80 - global_seqlen/min:29962 - global_seqlen/max:105447 - global_seqlen/minmax_diff:75485 - global_seqlen/balanced_min:60292 - global_seqlen/balanced_max:60692 - global_seqlen/mean:60551.375 - actor/entropy:0.37467148900032043 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3974228799343109 - training/rollout_probs_diff_mean:0.0030430646147578955 - training/rollout_probs_diff_std:0.007756151724606752 - training/rollout_actor_probs_pearson_corr:0.9996011257171631 - rollout_corr/rollout_is_mean:0.9999578595161438 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.408228510830668e-06 - rollout_corr/rollout_is_max:1.7704508304595947 - rollout_corr/rollout_is_min:0.40431222319602966 - rollout_corr/rollout_is_std:0.02627207152545452 - rollout_corr/rollout_is_eff_sample_size:0.9993102542910165 - rollout_corr/rollout_is_seq_mean:0.9999442100524902 - rollout_corr/rollout_is_seq_std:0.0010222762357443571 - rollout_corr/rollout_is_seq_max:1.0053536891937256 - rollout_corr/rollout_is_seq_min:0.9961909055709839 - rollout_corr/rollout_is_seq_max_deviation:0.005353689193725586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3535777013748884) - rollout_corr/training_log_ppl:np.float64(0.2919448420871049) - rollout_corr/kl:np.float64(0.0002914607577544359) - rollout_corr/k3_kl:np.float64(0.00035389692166631903) - rollout_corr/rollout_ppl:np.float64(1.3531782091595232) - rollout_corr/rollout_log_ppl:np.float64(0.2916533798270393) - rollout_corr/log_ppl_diff:np.float64(0.0002914622600656003) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007385123462881893) - rollout_corr/log_ppl_diff_max:np.float64(0.004037708044052124) - rollout_corr/log_ppl_diff_min:np.float64(-0.004292726516723633) - rollout_corr/ppl_ratio:np.float64(1.0002919645048678) - rollout_corr/chi2_token:np.float64(0.0008356906473636627) - rollout_corr/chi2_seq:np.float64(10.826386786764488) - actor/pg_loss:np.float64(2.2610998712480068e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.29967929121267e-05) - actor/ppo_kl:np.float64(-0.00010968260576760258) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08275915030390024) - perf/mfu/actor:np.float64(0.17912262687715078) - perf/max_memory_allocated_gb:np.float64(133.4042763710022) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.1332893371582) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7125) - val-aux/sciknoweval/reward/std@16:np.float64(0.21208410796593385) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.8003625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1838824791651191) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6199874999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1767685182330923) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7112875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21264807771425337) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8766124999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13289130750751482) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.551875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12414401859208643) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7451) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18095144568917293) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.9285499999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08117133451240169) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5055999999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0797599664446933) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7729874999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14284952333254633) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9576999999999998) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03905052666534377) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47392500000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0470622776131008) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7932874999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10751795426918813) - val-aux/sciknoweval/score/mean@16:np.float64(0.7125) - val-aux/sciknoweval/score/std@16:np.float64(0.21208410796593385) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.8003625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1838824791651191) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6199874999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1767685182330923) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7112875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21264807771425337) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8766124999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13289130750751482) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.551875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12414401859208643) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7451) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18095144568917293) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.9285499999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08117133451240169) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5055999999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0797599664446933) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7729874999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14284952333254633) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9576999999999998) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03905052666534377) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47392500000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0470622776131008) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7932874999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10751795426918813) - val-core/sciknoweval/acc/mean@16:np.float64(0.7125) - val-aux/sciknoweval/acc/std@16:np.float64(0.21208410796593385) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.8003625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1838824791651191) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6199874999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1767685182330923) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7112875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21264807771425337) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8766124999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13289130750751482) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.551875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12414401859208643) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7451) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18095144568917293) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.9285499999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08117133451240169) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5055999999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0797599664446933) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7729874999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14284952333254633) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9576999999999998) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03905052666534377) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47392500000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0470622776131008) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7932874999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10751795426918813) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.92890625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.11586940894478473) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.972475) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.06345428095007741) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.88225) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.13239906930031728) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.92675) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11753222858603023) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.990875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.02434920966995723) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8277999999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.13386162236149496) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.952825) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.09039223284283485) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9973500000000002) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.008420884420166629) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7695875) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11560936592748024) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9721375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.06158716693942362) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0019515618744994994) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.720375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08171985491238151) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9849) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03299085003096019) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013781989924609661 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013781989924609661 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1622.04296875 - response_length/max:8192.0 - response_length/min:248.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1622.04296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:248.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010387226939201355 - timing_s/agent_loop/generate_sequences/min:np.float64(2.926741849631071) - timing_s/agent_loop/generate_sequences/max:np.float64(69.41407606191933) - timing_s/agent_loop/generate_sequences/mean:np.float64(15.920171771431342) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.41407606191933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.53145934268832 - timing_s/reward:0.1108447965234518 - timing_s/old_log_prob:4.27453831396997 - timing_s/adv:0.18335014954209328 - timing_s/update_actor:18.237299781292677 - timing_s/step:99.4225641246885 - timing_s/testing:192.12527464702725 - timing_s/save_checkpoint:12.946154896169901 - timing_s/stop_profile:0.00015969760715961456 - timing_per_token_ms/gen:0.18430523655471212 - timing_per_token_ms/update_actor:0.03764840142212435 - timing_per_token_ms/adv:0.00037850120980343813 - perf/total_num_tokens:484411 - perf/time_per_step:99.4225641246885 - perf/throughput:609.0305106601445 (TaskRunner pid=2281424) Training Progress: 80%|████████ | 80/100 [1:47:49<53:06, 159.33s/it] (TaskRunner pid=2281424) step:81 - global_seqlen/min:47325 - global_seqlen/max:105881 - global_seqlen/minmax_diff:58556 - global_seqlen/balanced_min:76357 - global_seqlen/balanced_max:76567 - global_seqlen/mean:76471.5 - actor/entropy:0.46122434735298157 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2159644365310669 - training/rollout_probs_diff_mean:0.0033320188522338867 - training/rollout_probs_diff_std:0.00773474620655179 - training/rollout_actor_probs_pearson_corr:0.9996447563171387 - rollout_corr/rollout_is_mean:1.0000454187393188 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8291126489639282 - rollout_corr/rollout_is_min:0.5626341700553894 - rollout_corr/rollout_is_std:0.02797088772058487 - rollout_corr/rollout_is_eff_sample_size:0.9992181220349029 - rollout_corr/rollout_is_seq_mean:1.0000308752059937 - rollout_corr/rollout_is_seq_std:0.0010952734155580401 - rollout_corr/rollout_is_seq_max:1.0038789510726929 - rollout_corr/rollout_is_seq_min:0.9951615929603577 - rollout_corr/rollout_is_seq_max_deviation:0.004838407039642334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4837682861834764) - rollout_corr/training_log_ppl:np.float64(0.3760610534227453) - rollout_corr/kl:np.float64(0.0004281692851382246) - rollout_corr/k3_kl:np.float64(0.0004241228809291897) - rollout_corr/rollout_ppl:np.float64(1.483103048056364) - rollout_corr/rollout_log_ppl:np.float64(0.37563288104138337) - rollout_corr/log_ppl_diff:np.float64(0.00042817238136194646) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008447545405942947) - rollout_corr/log_ppl_diff_max:np.float64(0.007132202386856079) - rollout_corr/log_ppl_diff_min:np.float64(-0.004386395215988159) - rollout_corr/ppl_ratio:np.float64(1.0004289988428354) - rollout_corr/chi2_token:np.float64(0.0008449205197393894) - rollout_corr/chi2_seq:np.float64(8.882342195371166) - actor/pg_loss:np.float64(8.517643436789513e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.26423552603228e-05) - actor/ppo_kl:np.float64(4.755724401039885e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.05668603768572211) - perf/mfu/actor:np.float64(0.20224053663035063) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.5755615234375) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021595492959022522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021595492959022522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2130.390625 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.01953125 - response_length_non_aborted/mean:2130.390625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.01953125 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.419792771339417e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.191812541335821) - timing_s/agent_loop/generate_sequences/max:np.float64(73.02005716599524) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.716089782734343) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.02005716599524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:79.11253042705357 - timing_s/reward:0.1305206846445799 - timing_s/old_log_prob:5.078640753403306 - timing_s/adv:0.14080399833619595 - timing_s/update_actor:20.568581249564886 - timing_s/step:105.1306124124676 - timing_s/stop_profile:0.0001254640519618988 - timing_per_token_ms/gen:0.14505946390966587 - timing_per_token_ms/update_actor:0.033621318480683796 - timing_per_token_ms/adv:0.00023015763770848607 - perf/total_num_tokens:611772 - perf/time_per_step:105.1306124124676 - perf/throughput:727.395172968013 (TaskRunner pid=2281424) Training Progress: 81%|████████ | 81/100 [1:49:34<45:18, 143.08s/it] (TaskRunner pid=2281424) step:82 - global_seqlen/min:38375 - global_seqlen/max:108226 - global_seqlen/minmax_diff:69851 - global_seqlen/balanced_min:78960 - global_seqlen/balanced_max:79182 - global_seqlen/mean:79033.125 - actor/entropy:0.4809735119342804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40625 - training/rollout_probs_diff_mean:0.0033739369828253984 - training/rollout_probs_diff_std:0.0077420733869075775 - training/rollout_actor_probs_pearson_corr:0.99964839220047 - rollout_corr/rollout_is_mean:1.0000008344650269 - rollout_corr/rollout_is_ratio_fraction_high:1.7803699847718235e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.901850378606468e-06 - rollout_corr/rollout_is_max:3.1266798973083496 - rollout_corr/rollout_is_min:0.04171716049313545 - rollout_corr/rollout_is_std:0.028216995298862457 - rollout_corr/rollout_is_eff_sample_size:0.9992043155636828 - rollout_corr/rollout_is_seq_mean:1.0000370740890503 - rollout_corr/rollout_is_seq_std:0.0007998693035915494 - rollout_corr/rollout_is_seq_max:1.0032259225845337 - rollout_corr/rollout_is_seq_min:0.9969495534896851 - rollout_corr/rollout_is_seq_max_deviation:0.0032259225845336914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.447751184925437) - rollout_corr/training_log_ppl:np.float64(0.3510043573041912) - rollout_corr/kl:np.float64(0.0003127274083301046) - rollout_corr/k3_kl:np.float64(0.0003627495793239177) - rollout_corr/rollout_ppl:np.float64(1.4472675155848265) - rollout_corr/rollout_log_ppl:np.float64(0.3506916323385667) - rollout_corr/log_ppl_diff:np.float64(0.00031272496562451124) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007065368699841201) - rollout_corr/log_ppl_diff_max:np.float64(0.0035865604877471924) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028453469276428223) - rollout_corr/ppl_ratio:np.float64(1.0003131539560854) - rollout_corr/chi2_token:np.float64(0.000826970674097538) - rollout_corr/chi2_seq:np.float64(2.990220111096278) - actor/pg_loss:np.float64(-1.262675505131483e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.107881173671558e-05) - actor/ppo_kl:np.float64(-1.3940705438253431e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0625862805172801) - perf/mfu/actor:np.float64(0.20966418106155954) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.6513671875) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017938785254955292 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017938785254955292 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2194.06640625 - response_length/max:8192.0 - response_length/min:305.0 - response_length/clip_ratio:0.01171875 - response_length_non_aborted/mean:2194.06640625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:305.0 - response_length_non_aborted/clip_ratio:0.01171875 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.206309914588928e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.8525668401271105) - timing_s/agent_loop/generate_sequences/max:np.float64(73.49001097865403) - timing_s/agent_loop/generate_sequences/mean:np.float64(21.367409962149395) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.49001097865403) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:81.01238861493766 - timing_s/reward:0.1340006049722433 - timing_s/old_log_prob:5.036404974758625 - timing_s/adv:0.17358910478651524 - timing_s/update_actor:20.482164518907666 - timing_s/step:106.89824829623103 - timing_s/stop_profile:0.00012538954615592957 - timing_per_token_ms/gen:0.14423202603423949 - timing_per_token_ms/update_actor:0.032394904856203756 - timing_per_token_ms/adv:0.0002745511846876155 - perf/total_num_tokens:632265 - perf/time_per_step:106.89824829623103 - perf/throughput:739.3304030669183 (TaskRunner pid=2281424) Training Progress: 82%|████████▏ | 82/100 [1:51:21<39:40, 132.24s/it] (TaskRunner pid=2281424) step:83 - global_seqlen/min:46708 - global_seqlen/max:91538 - global_seqlen/minmax_diff:44830 - global_seqlen/balanced_min:69220 - global_seqlen/balanced_max:69538 - global_seqlen/mean:69476.25 - actor/entropy:0.5335648059844971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21911153197288513 - training/rollout_probs_diff_mean:0.0035760728642344475 - training/rollout_probs_diff_std:0.007884988561272621 - training/rollout_actor_probs_pearson_corr:0.9996548891067505 - rollout_corr/rollout_is_mean:1.0000134706497192 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9868483543395996 - rollout_corr/rollout_is_min:0.5721929669380188 - rollout_corr/rollout_is_std:0.029507704079151154 - rollout_corr/rollout_is_eff_sample_size:0.9991300528161801 - rollout_corr/rollout_is_seq_mean:0.9999951124191284 - rollout_corr/rollout_is_seq_std:0.0008592059020884335 - rollout_corr/rollout_is_seq_max:1.0029661655426025 - rollout_corr/rollout_is_seq_min:0.9966848492622375 - rollout_corr/rollout_is_seq_max_deviation:0.003315150737762451 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5849693166092038) - rollout_corr/training_log_ppl:np.float64(0.43137292834580876) - rollout_corr/kl:np.float64(0.0004295566333016865) - rollout_corr/k3_kl:np.float64(0.000427562826786243) - rollout_corr/rollout_ppl:np.float64(1.5842498331330717) - rollout_corr/rollout_log_ppl:np.float64(0.43094336806098) - rollout_corr/log_ppl_diff:np.float64(0.0004295602848287672) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007582434045616537) - rollout_corr/log_ppl_diff_max:np.float64(0.0037308931350708008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0021752119064331055) - rollout_corr/ppl_ratio:np.float64(1.000430047744885) - rollout_corr/chi2_token:np.float64(0.0008571695070713758) - rollout_corr/chi2_seq:np.float64(14.75534682860598) - actor/pg_loss:np.float64(5.458039231598377e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001093973785941671) - actor/ppo_kl:np.float64(3.278788307525815e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11635340005159378) - perf/mfu/actor:np.float64(0.19497208017666126) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.30697250366211) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005556248594075441 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005556248594075441 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1894.0703125 - response_length/max:7501.0 - response_length/min:209.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1894.0703125 - response_length_non_aborted/max:7501.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014759600162506104 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8615881502628326) - timing_s/agent_loop/generate_sequences/max:np.float64(58.530355628579855) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.284697317627433) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.530355628579855) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:7501 - timing_s/gen:66.29873481579125 - timing_s/reward:0.12525632046163082 - timing_s/old_log_prob:4.649687595665455 - timing_s/adv:0.13868866860866547 - timing_s/update_actor:19.150086453184485 - timing_s/step:90.4499344099313 - timing_s/stop_profile:0.0001360122114419937 - timing_per_token_ms/gen:0.13673168897956875 - timing_per_token_ms/update_actor:0.034454375511747695 - timing_per_token_ms/adv:0.00024952532089862626 - perf/total_num_tokens:555810 - perf/time_per_step:90.4499344099313 - perf/throughput:768.1183016134015 (TaskRunner pid=2281424) Training Progress: 83%|████████▎ | 83/100 [1:52:51<33:55, 119.72s/it] (TaskRunner pid=2281424) step:84 - global_seqlen/min:46333 - global_seqlen/max:120390 - global_seqlen/minmax_diff:74057 - global_seqlen/balanced_min:81591 - global_seqlen/balanced_max:81854 - global_seqlen/mean:81763.5 - actor/entropy:0.46362319588661194 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21383929252624512 - training/rollout_probs_diff_mean:0.003217406803742051 - training/rollout_probs_diff_std:0.007578108459711075 - training/rollout_actor_probs_pearson_corr:0.9996676445007324 - rollout_corr/rollout_is_mean:1.0000669956207275 - rollout_corr/rollout_is_ratio_fraction_high:3.410757472011028e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.705378736005514e-06 - rollout_corr/rollout_is_max:2.3830087184906006 - rollout_corr/rollout_is_min:0.15366080403327942 - rollout_corr/rollout_is_std:0.027716144919395447 - rollout_corr/rollout_is_eff_sample_size:0.9992325240192108 - rollout_corr/rollout_is_seq_mean:1.0001479387283325 - rollout_corr/rollout_is_seq_std:0.0008669063681736588 - rollout_corr/rollout_is_seq_max:1.0025583505630493 - rollout_corr/rollout_is_seq_min:0.9951901435852051 - rollout_corr/rollout_is_seq_max_deviation:0.004809856414794922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5030197105370462) - rollout_corr/training_log_ppl:np.float64(0.3850575773685705) - rollout_corr/kl:np.float64(0.00027459628167836314) - rollout_corr/k3_kl:np.float64(0.00038053442332852683) - rollout_corr/rollout_ppl:np.float64(1.5025763548910618) - rollout_corr/rollout_log_ppl:np.float64(0.38478297731489874) - rollout_corr/log_ppl_diff:np.float64(0.00027460005367174745) - rollout_corr/log_ppl_abs_diff:np.float64(0.0006311151082627475) - rollout_corr/log_ppl_diff_max:np.float64(0.005537629127502441) - rollout_corr/log_ppl_diff_min:np.float64(-0.002191781997680664) - rollout_corr/ppl_ratio:np.float64(1.0002749729901552) - rollout_corr/chi2_token:np.float64(0.0009795920923352242) - rollout_corr/chi2_seq:np.float64(5.221234666882083) - actor/pg_loss:np.float64(-1.2737466022372246e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.926910533091359e-05) - actor/ppo_kl:np.float64(4.087090245974423e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08847346901893616) - perf/mfu/actor:np.float64(0.21121760504386758) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.31388092041016) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.028147703036665916 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.028147703036665916 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:2290.546875 - response_length/max:8192.0 - response_length/min:261.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:2290.546875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:261.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.132176637649536e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.4179056379944086) - timing_s/agent_loop/generate_sequences/max:np.float64(73.46726548857987) - timing_s/agent_loop/generate_sequences/mean:np.float64(22.820737626556365) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.46726548857987) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:80.69648661091924 - timing_s/reward:0.13805697858333588 - timing_s/old_log_prob:5.1129378993064165 - timing_s/adv:0.1517045572400093 - timing_s/update_actor:20.95853779092431 - timing_s/step:107.14519646950066 - timing_s/stop_profile:0.0001168474555015564 - timing_per_token_ms/gen:0.13761807464599618 - timing_per_token_ms/update_actor:0.03204140262911371 - timing_per_token_ms/adv:0.0002319258551187408 - perf/total_num_tokens:654108 - perf/time_per_step:107.14519646950066 - perf/throughput:763.1093384879306 (TaskRunner pid=2281424) Training Progress: 84%|████████▍ | 84/100 [1:54:39<30:55, 115.97s/it] (TaskRunner pid=2281424) step:85 - global_seqlen/min:32778 - global_seqlen/max:118304 - global_seqlen/minmax_diff:85526 - global_seqlen/balanced_min:71971 - global_seqlen/balanced_max:72255 - global_seqlen/mean:72087.625 - actor/entropy:0.43251606822013855 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4627995491027832 - training/rollout_probs_diff_mean:0.0032046264968812466 - training/rollout_probs_diff_std:0.0077904630452394485 - training/rollout_actor_probs_pearson_corr:0.9996163845062256 - rollout_corr/rollout_is_mean:1.0000158548355103 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.962150236067828e-06 - rollout_corr/rollout_is_max:1.9618604183197021 - rollout_corr/rollout_is_min:0.2549246847629547 - rollout_corr/rollout_is_std:0.027508918195962906 - rollout_corr/rollout_is_eff_sample_size:0.9992438316531975 - rollout_corr/rollout_is_seq_mean:1.0000420808792114 - rollout_corr/rollout_is_seq_std:0.0008891079341992736 - rollout_corr/rollout_is_seq_max:1.0028163194656372 - rollout_corr/rollout_is_seq_min:0.9960147738456726 - rollout_corr/rollout_is_seq_max_deviation:0.003985226154327393 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.42215026775375) - rollout_corr/training_log_ppl:np.float64(0.33671512818546034) - rollout_corr/kl:np.float64(0.0003456190911617085) - rollout_corr/k3_kl:np.float64(0.00036502632008250657) - rollout_corr/rollout_ppl:np.float64(1.4216335988603532) - rollout_corr/rollout_log_ppl:np.float64(0.33636950654909015) - rollout_corr/log_ppl_diff:np.float64(0.00034562163637019694) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007374820706900209) - rollout_corr/log_ppl_diff_max:np.float64(0.004794985055923462) - rollout_corr/log_ppl_diff_min:np.float64(-0.00254783034324646) - rollout_corr/ppl_ratio:np.float64(1.000346165150404) - rollout_corr/chi2_token:np.float64(0.0007752671372145414) - rollout_corr/chi2_seq:np.float64(2.035842107143253) - actor/pg_loss:np.float64(-3.1908624805510044e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.119389224248152e-05) - actor/ppo_kl:np.float64(2.0877532892260575e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0993273975327611) - perf/mfu/actor:np.float64(0.19706691211589414) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.26990127563477) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01401048805564642 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01401048805564642 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1990.80078125 - response_length/max:8192.0 - response_length/min:199.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1990.80078125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.484799087047577e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4878016971051693) - timing_s/agent_loop/generate_sequences/max:np.float64(64.35746593214571) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.05917956525809) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(64.35746593214571) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:71.24950071051717 - timing_s/reward:0.11637670919299126 - timing_s/old_log_prob:4.78818198479712 - timing_s/adv:0.16108788177371025 - timing_s/update_actor:19.909558206796646 - timing_s/step:96.25776395574212 - timing_s/stop_profile:2.9178336262702942e-05 - timing_per_token_ms/gen:0.13980221666163148 - timing_per_token_ms/update_actor:0.034523190018391936 - timing_per_token_ms/adv:0.0002793265171617706 - perf/total_num_tokens:576701 - perf/time_per_step:96.25776395574212 - perf/throughput:748.9019278813167 (TaskRunner pid=2281424) Training Progress: 85%|████████▌ | 85/100 [1:56:15<27:30, 110.06s/it] (TaskRunner pid=2281424) step:86 - global_seqlen/min:29204 - global_seqlen/max:122423 - global_seqlen/minmax_diff:93219 - global_seqlen/balanced_min:71868 - global_seqlen/balanced_max:72149 - global_seqlen/mean:72006.125 - actor/entropy:0.447162926197052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48606076836586 - training/rollout_probs_diff_mean:0.003287011291831732 - training/rollout_probs_diff_std:0.007797359488904476 - training/rollout_actor_probs_pearson_corr:0.9996358752250671 - rollout_corr/rollout_is_mean:1.0000190734863281 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.949065103370231e-06 - rollout_corr/rollout_is_max:1.695367455482483 - rollout_corr/rollout_is_min:0.47366517782211304 - rollout_corr/rollout_is_std:0.02779559977352619 - rollout_corr/rollout_is_eff_sample_size:0.9992278820119855 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0009104225318878889 - rollout_corr/rollout_is_seq_max:1.0035185813903809 - rollout_corr/rollout_is_seq_min:0.9944294691085815 - rollout_corr/rollout_is_seq_max_deviation:0.005570530891418457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4478634255938232) - rollout_corr/training_log_ppl:np.float64(0.3556695947772823) - rollout_corr/kl:np.float64(0.00028983059334386496) - rollout_corr/k3_kl:np.float64(0.0003808262275128982) - rollout_corr/rollout_ppl:np.float64(1.4474190385080874) - rollout_corr/rollout_log_ppl:np.float64(0.35537976160412654) - rollout_corr/log_ppl_diff:np.float64(0.0002898331731557846) - rollout_corr/log_ppl_abs_diff:np.float64(0.0006754108471795917) - rollout_corr/log_ppl_diff_max:np.float64(0.006349921226501465) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031312406063079834) - rollout_corr/ppl_ratio:np.float64(1.000290306750685) - rollout_corr/chi2_token:np.float64(0.0009537497535347939) - rollout_corr/chi2_seq:np.float64(1.8716313727200031) - actor/pg_loss:np.float64(-4.055839963257313e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.612786794832573e-05) - actor/ppo_kl:np.float64(-2.927031754040854e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08380211982876062) - perf/mfu/actor:np.float64(0.19894113002162722) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.34819412231445) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012212483212351799 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012212483212351799 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1978.31640625 - response_length/max:7625.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1978.31640625 - response_length_non_aborted/max:7625.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6423484086990356e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2320269010961056) - timing_s/agent_loop/generate_sequences/max:np.float64(58.958134239539504) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.39341227179102) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.958134239539504) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:7625 - timing_s/gen:66.23730275034904 - timing_s/reward:0.1252636183053255 - timing_s/old_log_prob:4.83253344334662 - timing_s/adv:0.144756231456995 - timing_s/update_actor:19.455502839758992 - timing_s/step:90.88217270746827 - timing_s/stop_profile:0.00012291967868804932 - timing_per_token_ms/gen:0.13078770567292866 - timing_per_token_ms/update_actor:0.03377404151341117 - timing_per_token_ms/adv:0.00025129152460466906 - perf/total_num_tokens:576049 - perf/time_per_step:90.88217270746827 - perf/throughput:792.301975787633 (TaskRunner pid=2281424) Training Progress: 86%|████████▌ | 86/100 [1:57:46<24:20, 104.32s/it] (TaskRunner pid=2281424) step:87 - global_seqlen/min:43406 - global_seqlen/max:108646 - global_seqlen/minmax_diff:65240 - global_seqlen/balanced_min:77124 - global_seqlen/balanced_max:77249 - global_seqlen/mean:77219.0 - actor/entropy:0.47046366333961487 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28602373600006104 - training/rollout_probs_diff_mean:0.003333775093778968 - training/rollout_probs_diff_std:0.007726138457655907 - training/rollout_actor_probs_pearson_corr:0.9997732043266296 - rollout_corr/rollout_is_mean:1.0000189542770386 - rollout_corr/rollout_is_ratio_fraction_high:1.8291968899575295e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0640363693237305 - rollout_corr/rollout_is_min:0.6168027520179749 - rollout_corr/rollout_is_std:0.02792823687195778 - rollout_corr/rollout_is_eff_sample_size:0.9992206215230673 - rollout_corr/rollout_is_seq_mean:0.9999931454658508 - rollout_corr/rollout_is_seq_std:0.0008395964978262782 - rollout_corr/rollout_is_seq_max:1.0021971464157104 - rollout_corr/rollout_is_seq_min:0.9952844977378845 - rollout_corr/rollout_is_seq_max_deviation:0.0047155022621154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.507551294285804) - rollout_corr/training_log_ppl:np.float64(0.3906772773480043) - rollout_corr/kl:np.float64(0.0004402913548275933) - rollout_corr/k3_kl:np.float64(0.0004143168899304328) - rollout_corr/rollout_ppl:np.float64(1.5068849357776344) - rollout_corr/rollout_log_ppl:np.float64(0.39023698499659076) - rollout_corr/log_ppl_diff:np.float64(0.0004402923514135182) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007724418537691236) - rollout_corr/log_ppl_diff_max:np.float64(0.00550532341003418) - rollout_corr/log_ppl_diff_min:np.float64(-0.0020297616720199585) - rollout_corr/ppl_ratio:np.float64(1.0004408617969602) - rollout_corr/chi2_token:np.float64(0.0008024710696190596) - rollout_corr/chi2_seq:np.float64(1.297804655507207) - actor/pg_loss:np.float64(-6.791017949581146e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000107869764519819) - actor/ppo_kl:np.float64(5.49935483284969e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10479225032031536) - perf/mfu/actor:np.float64(0.20496773282402933) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.29073524475098) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003920197486877441 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003920197486877441 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2135.5 - response_length/max:7668.0 - response_length/min:246.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:2135.5 - response_length_non_aborted/max:7668.0 - response_length_non_aborted/min:246.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001415777951478958 - timing_s/agent_loop/generate_sequences/min:np.float64(3.136248679831624) - timing_s/agent_loop/generate_sequences/max:np.float64(58.29445764608681) - timing_s/agent_loop/generate_sequences/mean:np.float64(20.58206392160355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.29445764608681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:7668 - timing_s/gen:65.97938266210258 - timing_s/reward:0.13106519170105457 - timing_s/old_log_prob:4.976280787959695 - timing_s/adv:0.12150675244629383 - timing_s/update_actor:20.37396141141653 - timing_s/step:91.67027896083891 - timing_s/stop_profile:0.00010777637362480164 - timing_per_token_ms/gen:0.12068928284890573 - timing_per_token_ms/update_actor:0.03298081011703164 - timing_per_token_ms/adv:0.0001966917993730394 - perf/total_num_tokens:617752 - perf/time_per_step:91.67027896083891 - perf/throughput:842.3558963204156 (TaskRunner pid=2281424) Training Progress: 87%|████████▋ | 87/100 [1:59:18<21:47, 100.54s/it] (TaskRunner pid=2281424) step:88 - global_seqlen/min:36604 - global_seqlen/max:113322 - global_seqlen/minmax_diff:76718 - global_seqlen/balanced_min:70235 - global_seqlen/balanced_max:70460 - global_seqlen/mean:70355.125 - actor/entropy:0.45366615056991577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292335391044617 - training/rollout_probs_diff_mean:0.0033171819522976875 - training/rollout_probs_diff_std:0.007824771106243134 - training/rollout_actor_probs_pearson_corr:0.9996778964996338 - rollout_corr/rollout_is_mean:1.0 - rollout_corr/rollout_is_ratio_fraction_high:2.0178824797767447e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.0178824797767447e-06 - rollout_corr/rollout_is_max:4.592259407043457 - rollout_corr/rollout_is_min:0.19030924141407013 - rollout_corr/rollout_is_std:0.02784273773431778 - rollout_corr/rollout_is_eff_sample_size:0.9992253824874978 - rollout_corr/rollout_is_seq_mean:1.0000131130218506 - rollout_corr/rollout_is_seq_std:0.0010293829254806042 - rollout_corr/rollout_is_seq_max:1.0040936470031738 - rollout_corr/rollout_is_seq_min:0.9965201020240784 - rollout_corr/rollout_is_seq_max_deviation:0.004093647003173828 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4986823541112244) - rollout_corr/training_log_ppl:np.float64(0.38453248073346913) - rollout_corr/kl:np.float64(0.00033108328769060336) - rollout_corr/k3_kl:np.float64(0.00040329458698806775) - rollout_corr/rollout_ppl:np.float64(1.4981776834465563) - rollout_corr/rollout_log_ppl:np.float64(0.3842013974208385) - rollout_corr/log_ppl_diff:np.float64(0.0003310833126306534) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007660561241209507) - rollout_corr/log_ppl_diff_max:np.float64(0.004498302936553955) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034332573413848877) - rollout_corr/ppl_ratio:np.float64(1.0003316532820463) - rollout_corr/chi2_token:np.float64(0.0009584731888025999) - rollout_corr/chi2_seq:np.float64(4.4919839494396) - actor/pg_loss:np.float64(3.3170217648148537e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010095978973367892) - actor/ppo_kl:np.float64(-5.047958963944055e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0923737701959908) - perf/mfu/actor:np.float64(0.19545780170266114) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.3228530883789) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021646076813340187 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021646076813340187 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1935.81640625 - response_length/max:8192.0 - response_length/min:217.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1935.81640625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:217.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.361468255519867e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7692256066948175) - timing_s/agent_loop/generate_sequences/max:np.float64(67.97121464461088) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.578832377446815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(67.97121464461088) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:212 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:75.0351722035557 - timing_s/reward:0.1233233455568552 - timing_s/old_log_prob:4.722648134455085 - timing_s/adv:0.11689798347651958 - timing_s/update_actor:19.538657259196043 - timing_s/step:99.62376413680613 - timing_s/stop_profile:0.00011381879448890686 - timing_per_token_ms/gen:0.15141215895981328 - timing_per_token_ms/update_actor:0.034714346074994615 - timing_per_token_ms/adv:0.00020769272934366827 - perf/total_num_tokens:562841 - perf/time_per_step:99.62376413680613 - perf/throughput:706.2082587382101 (TaskRunner pid=2281424) Training Progress: 88%|████████▊ | 88/100 [2:00:57<20:03, 100.28s/it] (TaskRunner pid=2281424) step:89 - global_seqlen/min:53363 - global_seqlen/max:96134 - global_seqlen/minmax_diff:42771 - global_seqlen/balanced_min:73474 - global_seqlen/balanced_max:73759 - global_seqlen/mean:73580.375 - actor/entropy:0.5425375699996948 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3450472354888916 - training/rollout_probs_diff_mean:0.0035728770308196545 - training/rollout_probs_diff_std:0.007812545634806156 - training/rollout_actor_probs_pearson_corr:0.999666154384613 - rollout_corr/rollout_is_mean:1.0000165700912476 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.851049314107513e-06 - rollout_corr/rollout_is_max:1.8271527290344238 - rollout_corr/rollout_is_min:0.16427424550056458 - rollout_corr/rollout_is_std:0.029384229332208633 - rollout_corr/rollout_is_eff_sample_size:0.9991374309926928 - rollout_corr/rollout_is_seq_mean:0.9999986290931702 - rollout_corr/rollout_is_seq_std:0.0008472701301798224 - rollout_corr/rollout_is_seq_max:1.0029807090759277 - rollout_corr/rollout_is_seq_min:0.9957017302513123 - rollout_corr/rollout_is_seq_max_deviation:0.004298269748687744 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5710625080391765) - rollout_corr/training_log_ppl:np.float64(0.4208992576750461) - rollout_corr/kl:np.float64(0.0004893235657874584) - rollout_corr/k3_kl:np.float64(0.0004040306061483534) - rollout_corr/rollout_ppl:np.float64(1.5702631180174649) - rollout_corr/rollout_log_ppl:np.float64(0.42040993153932504) - rollout_corr/log_ppl_diff:np.float64(0.0004893261357210577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007742471643723547) - rollout_corr/log_ppl_diff_max:np.float64(0.0040569007396698) - rollout_corr/log_ppl_diff_min:np.float64(-0.0024844110012054443) - rollout_corr/ppl_ratio:np.float64(1.0004898344632238) - rollout_corr/chi2_token:np.float64(0.0006377112586051226) - rollout_corr/chi2_seq:np.float64(1.1348994506988674) - actor/pg_loss:np.float64(-6.997038144618273e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.692707725936998e-05) - actor/ppo_kl:np.float64(9.186447801567965e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0774675034917891) - perf/mfu/actor:np.float64(0.20087032175496733) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.02946090698242) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.029441511258482933 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.029441511258482933 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2028.66796875 - response_length/max:8192.0 - response_length/min:153.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:2028.66796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00038411468267440796 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9795979037880898) - timing_s/agent_loop/generate_sequences/max:np.float64(68.82650169543922) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.44817877533933) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(68.82650169543922) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.69867212511599 - timing_s/reward:0.12369080632925034 - timing_s/old_log_prob:4.9405558202415705 - timing_s/adv:0.16437353193759918 - timing_s/update_actor:19.898798257112503 - timing_s/step:101.944374775514 - timing_s/stop_profile:0.00012306682765483856 - timing_per_token_ms/gen:0.14768517697518574 - timing_per_token_ms/update_actor:0.03380452711934484 - timing_per_token_ms/adv:0.00027924146203658104 - perf/total_num_tokens:588643 - perf/time_per_step:101.944374775514 - perf/throughput:721.769839307242 (TaskRunner pid=2281424) Training Progress: 89%|████████▉ | 89/100 [2:02:40<18:30, 100.93s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] We are given the electric field $ E $ at a distance $ r $ from a point charge $ q $, and we are to find the magnitude of the charge. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are to solve for $ q $. Rearranging the formula: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Compute $ r^2 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) r^2 = (0.50 \, \mathrm{m})^2 = 0.25 \, \mathrm{m^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Plug into the formula (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \cdot (0.25 \, \mathrm{m^2})}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) Since $ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, we check the options: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - A: 50 pC (TaskRunner pid=2281424) - B: 56 pC (TaskRunner pid=2281424) - C: 60 pC (TaskRunner pid=2281424) - D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) Our calculated value is approximately **55.6 pC**, which is **closest to option B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We used the formula for the electric field due to a point charge, $ E = k \frac{q}{r^2} $, and rearranged it to solve for $ q $. We converted the given distance into meters, calculated $ r^2 $, and then used the values of $ E $ and $ k $ to compute the charge. We then converted the result into picocoulombs and matched it with the given options. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2281424) step:90 - global_seqlen/min:31582 - global_seqlen/max:87120 - global_seqlen/minmax_diff:55538 - global_seqlen/balanced_min:57199 - global_seqlen/balanced_max:57669 - global_seqlen/mean:57438.75 - actor/entropy:0.45833009481430054 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28802943229675293 - training/rollout_probs_diff_mean:0.0033582851756364107 - training/rollout_probs_diff_std:0.007876868359744549 - training/rollout_actor_probs_pearson_corr:0.9996347427368164 - rollout_corr/rollout_is_mean:1.0000334978103638 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.112814051244641e-06 - rollout_corr/rollout_is_max:1.816962718963623 - rollout_corr/rollout_is_min:0.38643011450767517 - rollout_corr/rollout_is_std:0.028212770819664 - rollout_corr/rollout_is_eff_sample_size:0.9992047916425459 - rollout_corr/rollout_is_seq_mean:1.0000895261764526 - rollout_corr/rollout_is_seq_std:0.0008514981018379331 - rollout_corr/rollout_is_seq_max:1.0030163526535034 - rollout_corr/rollout_is_seq_min:0.996926486492157 - rollout_corr/rollout_is_seq_max_deviation:0.0030735135078430176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4863704326562583) - rollout_corr/training_log_ppl:np.float64(0.3669586373725906) - rollout_corr/kl:np.float64(0.0003789341422653081) - rollout_corr/k3_kl:np.float64(0.0003786877977631775) - rollout_corr/rollout_ppl:np.float64(1.4857277404516935) - rollout_corr/rollout_log_ppl:np.float64(0.3665797036665026) - rollout_corr/log_ppl_diff:np.float64(0.0003789337060879916) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007170522294472903) - rollout_corr/log_ppl_diff_max:np.float64(0.0034637898206710815) - rollout_corr/log_ppl_diff_min:np.float64(-0.0026997029781341553) - rollout_corr/ppl_ratio:np.float64(1.000379371456802) - rollout_corr/chi2_token:np.float64(0.0007569114677608013) - rollout_corr/chi2_seq:np.float64(19.86091840430163) - actor/pg_loss:np.float64(-9.84076177701354e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.184421224337711e-05) - actor/ppo_kl:np.float64(0.00010069883839824989) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08238446805626154) - perf/mfu/actor:np.float64(0.17647744927476752) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.05528259277344) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.72265625) - val-aux/sciknoweval/reward/std@16:np.float64(0.2057458394439798) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.8098875000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17178028077323276) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6335) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17894301231008136) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.720275) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20532848626214437) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8814499999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11395992699905108) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5634875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13094232605191994) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7604375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17656682501213306) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.9239125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06209606071752287) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5146) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08708453812159662) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7867500000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14109642832776273) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9435625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.024301931121292062) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.48135000000000006) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.054652451285185435) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.8034125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11089874407322804) - val-aux/sciknoweval/score/mean@16:np.float64(0.72265625) - val-aux/sciknoweval/score/std@16:np.float64(0.2057458394439798) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.8098875000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17178028077323276) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6335) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17894301231008136) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.720275) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20532848626214437) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8814499999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11395992699905108) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5634875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13094232605191994) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7604375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17656682501213306) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.9239125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06209606071752287) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5146) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08708453812159662) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7867500000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14109642832776273) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9435625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.024301931121292062) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.48135000000000006) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.054652451285185435) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.8034125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11089874407322804) - val-core/sciknoweval/acc/mean@16:np.float64(0.72265625) - val-aux/sciknoweval/acc/std@16:np.float64(0.2057458394439798) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.8098875000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17178028077323276) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6335) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17894301231008136) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.720275) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20532848626214437) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8814499999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11395992699905108) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5634875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13094232605191994) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7604375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17656682501213306) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.9239125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06209606071752287) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5146) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08708453812159662) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7867500000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14109642832776273) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9435625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.024301931121292062) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.48135000000000006) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.054652451285185435) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.8034125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11089874407322804) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9390625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.1062386511391431) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9786999999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05641481524452378) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8982875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.12304830102207487) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9378250000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.10790044372990266) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.994575) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.018566940076233975) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8458500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12479054833478716) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9613875000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.08324671963700653) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9990874999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.004891743956974362) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.792075) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.10839186005437869) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9796250000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.05464609964655156) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.000683625445693766) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.745775) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0776266069855637) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.991225) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.02706783139141474) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017509790137410164 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017509790137410164 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1528.0234375 - response_length/max:8192.0 - response_length/min:305.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1528.0234375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:305.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001458972692489624 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6096901912242174) - timing_s/agent_loop/generate_sequences/max:np.float64(68.86912310123444) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.475064634047158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(68.86912310123444) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:256 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.93200786039233 - timing_s/reward:0.10610510408878326 - timing_s/old_log_prob:4.28859911672771 - timing_s/adv:0.1231590025126934 - timing_s/update_actor:17.333123948425055 - timing_s/step:98.87560060434043 - timing_s/testing:198.1766846049577 - timing_s/save_checkpoint:12.776142889633775 - timing_s/stop_profile:0.00014507584273815155 - timing_per_token_ms/gen:0.19666953289429343 - timing_per_token_ms/update_actor:0.03772088517861429 - timing_per_token_ms/adv:0.0002680224641742147 - perf/total_num_tokens:459510 - perf/time_per_step:98.87560060434043 - perf/throughput:580.919353702298 (TaskRunner pid=2281424) Training Progress: 90%|█████████ | 90/100 [2:07:49<27:16, 163.62s/it] (TaskRunner pid=2281424) step:91 - global_seqlen/min:30432 - global_seqlen/max:80533 - global_seqlen/minmax_diff:50101 - global_seqlen/balanced_min:59087 - global_seqlen/balanced_max:59360 - global_seqlen/mean:59265.0 - actor/entropy:0.4014686048030853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6135859489440918 - training/rollout_probs_diff_mean:0.0031206756830215454 - training/rollout_probs_diff_std:0.007770925760269165 - training/rollout_actor_probs_pearson_corr:0.9996281266212463 - rollout_corr/rollout_is_mean:1.000016450881958 - rollout_corr/rollout_is_ratio_fraction_high:2.4419784949714085e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.8256072998046875 - rollout_corr/rollout_is_min:0.535260796546936 - rollout_corr/rollout_is_std:0.026806596666574478 - rollout_corr/rollout_is_eff_sample_size:0.9992819224085475 - rollout_corr/rollout_is_seq_mean:1.0000858306884766 - rollout_corr/rollout_is_seq_std:0.0011405966943129897 - rollout_corr/rollout_is_seq_max:1.0064619779586792 - rollout_corr/rollout_is_seq_min:0.9956552982330322 - rollout_corr/rollout_is_seq_max_deviation:0.006461977958679199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3793772477656603) - rollout_corr/training_log_ppl:np.float64(0.3077376027358696) - rollout_corr/kl:np.float64(0.0003082244062286854) - rollout_corr/k3_kl:np.float64(0.00036910291433400744) - rollout_corr/rollout_ppl:np.float64(1.3789437217637897) - rollout_corr/rollout_log_ppl:np.float64(0.3074293795507401) - rollout_corr/log_ppl_diff:np.float64(0.00030822318512946367) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008478707168251276) - rollout_corr/log_ppl_diff_max:np.float64(0.00515410304069519) - rollout_corr/log_ppl_diff_min:np.float64(-0.00571519136428833) - rollout_corr/ppl_ratio:np.float64(1.00030901376158) - rollout_corr/chi2_token:np.float64(0.000863607507199049) - rollout_corr/chi2_seq:np.float64(2.677014630055055) - actor/pg_loss:np.float64(8.078990504145622e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.9724320180976065e-05) - actor/ppo_kl:np.float64(3.992081970327632e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.056282463716343045) - perf/mfu/actor:np.float64(0.17517399995117178) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.51257610321045) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017548669129610062 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017548669129610062 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1599.625 - response_length/max:8192.0 - response_length/min:181.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1599.625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001323893666267395 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8850416410714388) - timing_s/agent_loop/generate_sequences/max:np.float64(68.87393903173506) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.232906831581204) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(68.87393903173506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:75.83647374995053 - timing_s/reward:0.10788371786475182 - timing_s/old_log_prob:4.422369368374348 - timing_s/adv:0.13758745603263378 - timing_s/update_actor:18.241979939863086 - timing_s/step:98.84038063883781 - timing_s/stop_profile:0.00011936575174331665 - timing_per_token_ms/gen:0.18519104514229537 - timing_per_token_ms/update_actor:0.038475449126514565 - timing_per_token_ms/adv:0.00029019542738680876 - perf/total_num_tokens:474120 - perf/time_per_step:98.84038063883781 - perf/throughput:599.6031137977299 (TaskRunner pid=2281424) Training Progress: 91%|█████████ | 91/100 [2:09:28<21:37, 144.20s/it] (TaskRunner pid=2281424) step:92 - global_seqlen/min:56361 - global_seqlen/max:133413 - global_seqlen/minmax_diff:77052 - global_seqlen/balanced_min:93215 - global_seqlen/balanced_max:93343 - global_seqlen/mean:93298.75 - actor/entropy:0.5294249653816223 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932349443435669 - training/rollout_probs_diff_mean:0.0034848994109779596 - training/rollout_probs_diff_std:0.0077215456403791904 - training/rollout_actor_probs_pearson_corr:0.9996566772460938 - rollout_corr/rollout_is_mean:1.0000396966934204 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.548328161239624 - rollout_corr/rollout_is_min:0.5352661609649658 - rollout_corr/rollout_is_std:0.028887107968330383 - rollout_corr/rollout_is_eff_sample_size:0.9991662307244454 - rollout_corr/rollout_is_seq_mean:0.999986469745636 - rollout_corr/rollout_is_seq_std:0.000814978324342519 - rollout_corr/rollout_is_seq_max:1.0046956539154053 - rollout_corr/rollout_is_seq_min:0.9965773224830627 - rollout_corr/rollout_is_seq_max_deviation:0.0046956539154052734 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.608520980924368) - rollout_corr/training_log_ppl:np.float64(0.45163694545044564) - rollout_corr/kl:np.float64(0.0004835717547688745) - rollout_corr/k3_kl:np.float64(0.00041193552203822037) - rollout_corr/rollout_ppl:np.float64(1.6077243499457836) - rollout_corr/rollout_log_ppl:np.float64(0.45115337034803815) - rollout_corr/log_ppl_diff:np.float64(0.00048357510240748525) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007026950479485095) - rollout_corr/log_ppl_diff_max:np.float64(0.003002941608428955) - rollout_corr/log_ppl_diff_min:np.float64(-0.0039307475090026855) - rollout_corr/ppl_ratio:np.float64(1.0004840346518904) - rollout_corr/chi2_token:np.float64(0.0006826643366366625) - rollout_corr/chi2_seq:np.float64(2.7582137731369585) - actor/pg_loss:np.float64(-2.270552795380354e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.691421706113033e-05) - actor/ppo_kl:np.float64(6.0937326300969374e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06705359183251858) - perf/mfu/actor:np.float64(0.22324610163002523) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.4158935546875) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004625823814421892 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004625823814421892 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:2637.3046875 - response_length/max:8192.0 - response_length/min:370.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:2637.3046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:370.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.772313594818115e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(4.7102756798267365) - timing_s/agent_loop/generate_sequences/max:np.float64(70.24139930866659) - timing_s/agent_loop/generate_sequences/mean:np.float64(26.584116022888338) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(70.24139930866659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:77.78476001881063 - timing_s/reward:0.14629439264535904 - timing_s/old_log_prob:5.78052232414484 - timing_s/adv:0.15388775803148746 - timing_s/update_actor:22.8142747040838 - timing_s/step:106.79661073349416 - timing_s/stop_profile:0.0001097414642572403 - timing_per_token_ms/gen:0.11521107904733856 - timing_per_token_ms/update_actor:0.030566158046173984 - timing_per_token_ms/adv:0.00020617607153296195 - perf/total_num_tokens:746390 - perf/time_per_step:106.79661073349416 - perf/throughput:873.6115253022642 (TaskRunner pid=2281424) Training Progress: 92%|█████████▏| 92/100 [2:11:15<17:43, 132.99s/it] (TaskRunner pid=2281424) step:93 - global_seqlen/min:32833 - global_seqlen/max:94956 - global_seqlen/minmax_diff:62123 - global_seqlen/balanced_min:68640 - global_seqlen/balanced_max:68729 - global_seqlen/mean:68692.75 - actor/entropy:0.5789089202880859 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18479880690574646 - training/rollout_probs_diff_mean:0.003710611490532756 - training/rollout_probs_diff_std:0.00791421253234148 - training/rollout_actor_probs_pearson_corr:0.9996806979179382 - rollout_corr/rollout_is_mean:0.9999197721481323 - rollout_corr/rollout_is_ratio_fraction_high:2.0674228835559916e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.97680926322937 - rollout_corr/rollout_is_min:0.6035233736038208 - rollout_corr/rollout_is_std:0.029875071719288826 - rollout_corr/rollout_is_eff_sample_size:0.9991082759306559 - rollout_corr/rollout_is_seq_mean:0.9998905658721924 - rollout_corr/rollout_is_seq_std:0.0011505308793857694 - rollout_corr/rollout_is_seq_max:1.0048060417175293 - rollout_corr/rollout_is_seq_min:0.994641125202179 - rollout_corr/rollout_is_seq_max_deviation:0.005358874797821045 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.601950139272958) - rollout_corr/training_log_ppl:np.float64(0.43084850793820806) - rollout_corr/kl:np.float64(0.0005708589492365856) - rollout_corr/k3_kl:np.float64(0.0004376700993589111) - rollout_corr/rollout_ppl:np.float64(1.600947569590062) - rollout_corr/rollout_log_ppl:np.float64(0.4302776500408072) - rollout_corr/log_ppl_diff:np.float64(0.000570857897400856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009169939439743757) - rollout_corr/log_ppl_diff_max:np.float64(0.005955815315246582) - rollout_corr/log_ppl_diff_min:np.float64(-0.002580583095550537) - rollout_corr/ppl_ratio:np.float64(1.0005716276355088) - rollout_corr/chi2_token:np.float64(0.0006089142989367247) - rollout_corr/chi2_seq:np.float64(0.175237572286278) - actor/pg_loss:np.float64(-2.8330832719802856e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.197003153829428e-05) - actor/ppo_kl:np.float64(2.893346624155413e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.057069569593295455) - perf/mfu/actor:np.float64(0.19020234349041532) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.21023941040039) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008691961877048016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008691961877048016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1889.4296875 - response_length/max:8192.0 - response_length/min:174.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1889.4296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002087317407131195 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1920957434922457) - timing_s/agent_loop/generate_sequences/max:np.float64(69.09191367775202) - timing_s/agent_loop/generate_sequences/mean:np.float64(17.2113787043927) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.09191367775202) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.51329581439495 - timing_s/reward:0.12370793335139751 - timing_s/old_log_prob:4.726794863119721 - timing_s/adv:0.12588824145495892 - timing_s/update_actor:19.58357641287148 - timing_s/step:101.16370816528797 - timing_s/stop_profile:8.470378816127777e-05 - timing_per_token_ms/gen:0.15818533166505053 - timing_per_token_ms/update_actor:0.03563617778599539 - timing_per_token_ms/adv:0.0002290784716272076 - perf/total_num_tokens:549542 - perf/time_per_step:101.16370816528797 - perf/throughput:679.0256233763716 (TaskRunner pid=2281424) Training Progress: 93%|█████████▎| 93/100 [2:12:56<14:24, 123.46s/it] (TaskRunner pid=2281424) step:94 - global_seqlen/min:33314 - global_seqlen/max:97152 - global_seqlen/minmax_diff:63838 - global_seqlen/balanced_min:55347 - global_seqlen/balanced_max:55703 - global_seqlen/mean:55618.375 - actor/entropy:0.44463133811950684 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5926223993301392 - training/rollout_probs_diff_mean:0.00327357673086226 - training/rollout_probs_diff_std:0.007880967110395432 - training/rollout_actor_probs_pearson_corr:0.9996113777160645 - rollout_corr/rollout_is_mean:1.0000139474868774 - rollout_corr/rollout_is_ratio_fraction_high:2.65053017756145e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3252651115180925e-05 - rollout_corr/rollout_is_max:2.4800922870635986 - rollout_corr/rollout_is_min:1.9368031644262373e-05 - rollout_corr/rollout_is_std:0.027902614325284958 - rollout_corr/rollout_is_eff_sample_size:0.9992220498076327 - rollout_corr/rollout_is_seq_mean:1.0000684261322021 - rollout_corr/rollout_is_seq_std:0.000999571057036519 - rollout_corr/rollout_is_seq_max:1.0037225484848022 - rollout_corr/rollout_is_seq_min:0.9954401254653931 - rollout_corr/rollout_is_seq_max_deviation:0.004559874534606934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4411272373981774) - rollout_corr/training_log_ppl:np.float64(0.34707161964615807) - rollout_corr/kl:np.float64(0.0003712050064246242) - rollout_corr/k3_kl:np.float64(0.0004094531111888955) - rollout_corr/rollout_ppl:np.float64(1.4405805715359747) - rollout_corr/rollout_log_ppl:np.float64(0.3467004152480513) - rollout_corr/log_ppl_diff:np.float64(0.00037120439810678363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007890164270065725) - rollout_corr/log_ppl_diff_max:np.float64(0.0038557052612304688) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032331347465515137) - rollout_corr/ppl_ratio:np.float64(1.000371773261577) - rollout_corr/chi2_token:np.float64(0.0008935551159083843) - rollout_corr/chi2_seq:np.float64(0.36217507324181497) - actor/pg_loss:np.float64(6.495625711977482e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.518509802484914e-05) - actor/ppo_kl:np.float64(3.6960312083778035e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.057536555686965585) - perf/mfu/actor:np.float64(0.17357501625996624) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.26631164550781) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010005089454352856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010005089454352856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1473.76171875 - response_length/max:7020.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1473.76171875 - response_length_non_aborted/max:7020.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.296353578567505e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.030196225270629) - timing_s/agent_loop/generate_sequences/max:np.float64(52.45747355185449) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.197725277575955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(52.45747355185449) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:7020 - timing_s/gen:59.61554795503616 - timing_s/reward:0.10644834861159325 - timing_s/old_log_prob:4.186762785539031 - timing_s/adv:0.14022722467780113 - timing_s/update_actor:17.202740924432874 - timing_s/step:81.33951114118099 - timing_s/stop_profile:0.00011213496327400208 - timing_per_token_ms/gen:0.15801281254399527 - timing_per_token_ms/update_actor:0.038662449515184674 - timing_per_token_ms/adv:0.0003151548941285167 - perf/total_num_tokens:444947 - perf/time_per_step:81.33951114118099 - perf/throughput:683.7805418262618 (TaskRunner pid=2281424) Training Progress: 94%|█████████▍| 94/100 [2:14:18<11:05, 110.84s/it] (TaskRunner pid=2281424) step:95 - global_seqlen/min:41511 - global_seqlen/max:154263 - global_seqlen/minmax_diff:112752 - global_seqlen/balanced_min:88799 - global_seqlen/balanced_max:88957 - global_seqlen/mean:88866.125 - actor/entropy:0.4982995390892029 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2708369493484497 - training/rollout_probs_diff_mean:0.00339501746930182 - training/rollout_probs_diff_std:0.007665873970836401 - training/rollout_actor_probs_pearson_corr:0.999656081199646 - rollout_corr/rollout_is_mean:1.000037670135498 - rollout_corr/rollout_is_ratio_fraction_high:1.5620679505445878e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.5386104583740234 - rollout_corr/rollout_is_min:0.503195583820343 - rollout_corr/rollout_is_std:0.02833504043519497 - rollout_corr/rollout_is_eff_sample_size:0.9991978885434327 - rollout_corr/rollout_is_seq_mean:1.000043511390686 - rollout_corr/rollout_is_seq_std:0.0009036674164235592 - rollout_corr/rollout_is_seq_max:1.0054752826690674 - rollout_corr/rollout_is_seq_min:0.9958840012550354 - rollout_corr/rollout_is_seq_max_deviation:0.005475282669067383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.540353175252676) - rollout_corr/training_log_ppl:np.float64(0.41098539560334757) - rollout_corr/kl:np.float64(0.0004560288263313339) - rollout_corr/k3_kl:np.float64(0.0003980033748689493) - rollout_corr/rollout_ppl:np.float64(1.5396650568582118) - rollout_corr/rollout_log_ppl:np.float64(0.41052936331834644) - rollout_corr/log_ppl_diff:np.float64(0.0004560322850011289) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007419722969643772) - rollout_corr/log_ppl_diff_max:np.float64(0.005006879568099976) - rollout_corr/log_ppl_diff_min:np.float64(-0.004607051610946655) - rollout_corr/ppl_ratio:np.float64(1.0004565534181893) - rollout_corr/chi2_token:np.float64(0.0006959009915590286) - rollout_corr/chi2_seq:np.float64(27.76160763693042) - actor/pg_loss:np.float64(4.626181907951832e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.550116794798669e-05) - actor/ppo_kl:np.float64(0.00011306502917474148) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06149573251605034) - perf/mfu/actor:np.float64(0.21980245524914177) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.26933860778809) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004443692974746227 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004443692974746227 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2500.69140625 - response_length/max:8192.0 - response_length/min:209.0 - response_length/clip_ratio:0.01953125 - response_length_non_aborted/mean:2500.69140625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.01953125 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010070018470287323 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8333873972296715) - timing_s/agent_loop/generate_sequences/max:np.float64(72.34981802105904) - timing_s/agent_loop/generate_sequences/mean:np.float64(24.544051925644453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(72.34981802105904) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:79.49770486913621 - timing_s/reward:0.14676685817539692 - timing_s/old_log_prob:5.38677878677845 - timing_s/adv:0.12247935496270657 - timing_s/update_actor:22.04177999868989 - timing_s/step:107.2831785902381 - timing_s/stop_profile:0.000116763636469841 - timing_per_token_ms/gen:0.12418082009996644 - timing_per_token_ms/update_actor:0.03100419310323519 - timing_per_token_ms/adv:0.00017228071293013306 - perf/total_num_tokens:710929 - perf/time_per_step:107.2831785902381 - perf/throughput:828.332327283283 (TaskRunner pid=2281424) Training Progress: 95%|█████████▌| 95/100 [2:16:05<09:08, 109.79s/it] (TaskRunner pid=2281424) step:96 - global_seqlen/min:36997 - global_seqlen/max:166139 - global_seqlen/minmax_diff:129142 - global_seqlen/balanced_min:75846 - global_seqlen/balanced_max:76107 - global_seqlen/mean:75997.125 - actor/entropy:0.471091628074646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2679024338722229 - training/rollout_probs_diff_mean:0.00336635485291481 - training/rollout_probs_diff_std:0.0077355667017400265 - training/rollout_actor_probs_pearson_corr:0.999660849571228 - rollout_corr/rollout_is_mean:0.9999645948410034 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.7283869005477754e-06 - rollout_corr/rollout_is_max:1.7948318719863892 - rollout_corr/rollout_is_min:0.12595275044441223 - rollout_corr/rollout_is_std:0.028181063011288643 - rollout_corr/rollout_is_eff_sample_size:0.9992062198818573 - rollout_corr/rollout_is_seq_mean:0.9999157190322876 - rollout_corr/rollout_is_seq_std:0.0007819948368705809 - rollout_corr/rollout_is_seq_max:1.0023274421691895 - rollout_corr/rollout_is_seq_min:0.9979436993598938 - rollout_corr/rollout_is_seq_max_deviation:0.002327442169189453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4727759365923703) - rollout_corr/training_log_ppl:np.float64(0.37059994105948135) - rollout_corr/kl:np.float64(0.00044277470152032095) - rollout_corr/k3_kl:np.float64(0.00038263250905856694) - rollout_corr/rollout_ppl:np.float64(1.4721139534376562) - rollout_corr/rollout_log_ppl:np.float64(0.3701571657729801) - rollout_corr/log_ppl_diff:np.float64(0.0004427752865012735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007409804675262421) - rollout_corr/log_ppl_diff_max:np.float64(0.002924293279647827) - rollout_corr/log_ppl_diff_min:np.float64(-0.001986294984817505) - rollout_corr/ppl_ratio:np.float64(1.0004432362038642) - rollout_corr/chi2_token:np.float64(0.00064484728500247) - rollout_corr/chi2_seq:np.float64(1.0331782191060483) - actor/pg_loss:np.float64(3.8074213080108166e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.83946193855445e-05) - actor/ppo_kl:np.float64(-1.7157419575752897e-05) - actor/pg_clipfrac_lower:np.float64(6.802943062211853e-07) - actor/grad_norm:np.float64(0.08832370489835739) - perf/mfu/actor:np.float64(0.20191487584379592) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.39874267578125) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016185393556952477 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.016185393556952477 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2095.41015625 - response_length/max:8192.0 - response_length/min:297.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:2095.41015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:297.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.160566747188568e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.8253472223877907) - timing_s/agent_loop/generate_sequences/max:np.float64(69.03189334087074) - timing_s/agent_loop/generate_sequences/mean:np.float64(20.723504332338052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.03189334087074) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:355 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.10101301595569 - timing_s/reward:0.13098066858947277 - timing_s/old_log_prob:4.933196693658829 - timing_s/adv:0.12136180512607098 - timing_s/update_actor:20.294502025470138 - timing_s/step:101.66740370728076 - timing_s/stop_profile:3.110058605670929e-05 - timing_per_token_ms/gen:0.1418670140578006 - timing_per_token_ms/update_actor:0.03338037791803002 - timing_per_token_ms/adv:0.00019961578337021133 - perf/total_num_tokens:607977 - perf/time_per_step:101.66740370728076 - perf/throughput:747.5072858043053 (TaskRunner pid=2281424) Training Progress: 96%|█████████▌| 96/100 [2:17:47<07:09, 107.36s/it] (TaskRunner pid=2281424) step:97 - global_seqlen/min:33062 - global_seqlen/max:136692 - global_seqlen/minmax_diff:103630 - global_seqlen/balanced_min:83301 - global_seqlen/balanced_max:83461 - global_seqlen/mean:83356.5 - actor/entropy:0.5467954874038696 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25347256660461426 - training/rollout_probs_diff_mean:0.0037388440687209368 - training/rollout_probs_diff_std:0.007841377519071102 - training/rollout_actor_probs_pearson_corr:0.9996518492698669 - rollout_corr/rollout_is_mean:1.000118374824524 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.344011929584667e-06 - rollout_corr/rollout_is_max:1.951640248298645 - rollout_corr/rollout_is_min:0.06364211440086365 - rollout_corr/rollout_is_std:0.029398424550890923 - rollout_corr/rollout_is_eff_sample_size:0.9991367169708485 - rollout_corr/rollout_is_seq_mean:1.0000510215759277 - rollout_corr/rollout_is_seq_std:0.00096407305682078 - rollout_corr/rollout_is_seq_max:1.0034452676773071 - rollout_corr/rollout_is_seq_min:0.9948387145996094 - rollout_corr/rollout_is_seq_max_deviation:0.005161285400390625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5564615204930305) - rollout_corr/training_log_ppl:np.float64(0.42052613582927734) - rollout_corr/kl:np.float64(0.000411179766622638) - rollout_corr/k3_kl:np.float64(0.00041175296956907914) - rollout_corr/rollout_ppl:np.float64(1.5558123434893787) - rollout_corr/rollout_log_ppl:np.float64(0.42011495429323986) - rollout_corr/log_ppl_diff:np.float64(0.00041118153603747487) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007968471036292613) - rollout_corr/log_ppl_diff_max:np.float64(0.005975067615509033) - rollout_corr/log_ppl_diff_min:np.float64(-0.0025658011436462402) - rollout_corr/ppl_ratio:np.float64(1.0004117703065276) - rollout_corr/chi2_token:np.float64(0.000819820212200284) - rollout_corr/chi2_seq:np.float64(4.5130012354347855) - actor/pg_loss:np.float64(1.147331204265356e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.38316911426773e-05) - actor/ppo_kl:np.float64(6.123169056237998e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09202379267662764) - perf/mfu/actor:np.float64(0.2117913412574436) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.30643844604492) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.039547327905893326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.039547327905893326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2336.265625 - response_length/max:8192.0 - response_length/min:270.0 - response_length/clip_ratio:0.0546875 - response_length_non_aborted/mean:2336.265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:270.0 - response_length_non_aborted/clip_ratio:0.0546875 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.753873407840729e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.633798986673355) - timing_s/agent_loop/generate_sequences/max:np.float64(78.11388960666955) - timing_s/agent_loop/generate_sequences/mean:np.float64(22.929097295367683) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(78.11388960666955) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:85.37431517802179 - timing_s/reward:0.1406618170440197 - timing_s/old_log_prob:5.179867882281542 - timing_s/adv:0.11821933090686798 - timing_s/update_actor:21.53932829387486 - timing_s/step:112.4388401247561 - timing_s/stop_profile:2.911873161792755e-05 - timing_per_token_ms/gen:0.1427463620127303 - timing_per_token_ms/update_actor:0.03230001303718795 - timing_per_token_ms/adv:0.0001772797126001991 - perf/total_num_tokens:666852 - perf/time_per_step:112.4388401247561 - perf/throughput:741.3496964884386 (TaskRunner pid=2281424) Training Progress: 97%|█████████▋| 97/100 [2:19:39<05:26, 108.89s/it] (TaskRunner pid=2281424) step:98 - global_seqlen/min:36746 - global_seqlen/max:101541 - global_seqlen/minmax_diff:64795 - global_seqlen/balanced_min:66846 - global_seqlen/balanced_max:67095 - global_seqlen/mean:66942.25 - actor/entropy:0.46268388628959656 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19511407613754272 - training/rollout_probs_diff_mean:0.0033976633567363024 - training/rollout_probs_diff_std:0.007802109699696302 - training/rollout_actor_probs_pearson_corr:0.9996366500854492 - rollout_corr/rollout_is_mean:1.000012755393982 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.758634090423584 - rollout_corr/rollout_is_min:0.5404731035232544 - rollout_corr/rollout_is_std:0.02801772952079773 - rollout_corr/rollout_is_eff_sample_size:0.9992156225592431 - rollout_corr/rollout_is_seq_mean:1.0000487565994263 - rollout_corr/rollout_is_seq_std:0.0007411489496007562 - rollout_corr/rollout_is_seq_max:1.0029963254928589 - rollout_corr/rollout_is_seq_min:0.9981603622436523 - rollout_corr/rollout_is_seq_max_deviation:0.0029963254928588867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4537494746036828) - rollout_corr/training_log_ppl:np.float64(0.3591947711247485) - rollout_corr/kl:np.float64(0.00034263647683197007) - rollout_corr/k3_kl:np.float64(0.0003613955028072269) - rollout_corr/rollout_ppl:np.float64(1.453213640023023) - rollout_corr/rollout_log_ppl:np.float64(0.35885213530855253) - rollout_corr/log_ppl_diff:np.float64(0.0003426358161959797) - rollout_corr/log_ppl_abs_diff:np.float64(0.0006818583060521632) - rollout_corr/log_ppl_diff_max:np.float64(0.0025803744792938232) - rollout_corr/log_ppl_diff_min:np.float64(-0.0025287121534347534) - rollout_corr/ppl_ratio:np.float64(1.0003430175129324) - rollout_corr/chi2_token:np.float64(0.0007636058144271374) - rollout_corr/chi2_seq:np.float64(1.1469660245347768) - actor/pg_loss:np.float64(3.07583250105381e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.736449529398669e-05) - actor/ppo_kl:np.float64(3.588981383906287e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06043858523480594) - perf/mfu/actor:np.float64(0.19140438179546104) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.1692008972168) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015156281180679798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015156281180679798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1822.2578125 - response_length/max:8192.0 - response_length/min:417.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1822.2578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:417.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.0846487283706665e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(5.219669699668884) - timing_s/agent_loop/generate_sequences/max:np.float64(69.51818436384201) - timing_s/agent_loop/generate_sequences/mean:np.float64(17.624326372162614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.51818436384201) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.9898233152926 - timing_s/reward:0.12057511694729328 - timing_s/old_log_prob:4.5568945575505495 - timing_s/adv:0.12083770520985126 - timing_s/update_actor:18.797641713172197 - timing_s/step:100.67088041082025 - timing_s/stop_profile:3.077089786529541e-05 - timing_per_token_ms/gen:0.16503784221002576 - timing_per_token_ms/update_actor:0.03510048159639876 - timing_per_token_ms/adv:0.00022563796632517444 - perf/total_num_tokens:535538 - perf/time_per_step:100.67088041082025 - perf/throughput:664.9614041997089 (TaskRunner pid=2281424) Training Progress: 98%|█████████▊| 98/100 [2:21:20<03:32, 106.43s/it] (TaskRunner pid=2281424) step:99 - global_seqlen/min:29160 - global_seqlen/max:172289 - global_seqlen/minmax_diff:143129 - global_seqlen/balanced_min:67431 - global_seqlen/balanced_max:67700 - global_seqlen/mean:67597.625 - actor/entropy:0.47339898347854614 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3376702070236206 - training/rollout_probs_diff_mean:0.003389441641047597 - training/rollout_probs_diff_std:0.007820799946784973 - training/rollout_actor_probs_pearson_corr:0.9996418952941895 - rollout_corr/rollout_is_mean:1.0000227689743042 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.224498297844548e-06 - rollout_corr/rollout_is_max:1.6443817615509033 - rollout_corr/rollout_is_min:0.14922289550304413 - rollout_corr/rollout_is_std:0.028145084157586098 - rollout_corr/rollout_is_eff_sample_size:0.9992084812691172 - rollout_corr/rollout_is_seq_mean:1.0001269578933716 - rollout_corr/rollout_is_seq_std:0.0011187855852767825 - rollout_corr/rollout_is_seq_max:1.0088918209075928 - rollout_corr/rollout_is_seq_min:0.9973447322845459 - rollout_corr/rollout_is_seq_max_deviation:0.008891820907592773 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4072618028149009) - rollout_corr/training_log_ppl:np.float64(0.3233967706328258) - rollout_corr/kl:np.float64(0.00018692354644933573) - rollout_corr/k3_kl:np.float64(0.00037853698717071893) - rollout_corr/rollout_ppl:np.float64(1.406971170566976) - rollout_corr/rollout_log_ppl:np.float64(0.3232098486623727) - rollout_corr/log_ppl_diff:np.float64(0.00018692197045311332) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007679337286390364) - rollout_corr/log_ppl_diff_max:np.float64(0.0030442774295806885) - rollout_corr/log_ppl_diff_min:np.float64(-0.007332652807235718) - rollout_corr/ppl_ratio:np.float64(1.000187524361536) - rollout_corr/chi2_token:np.float64(0.00115422741509974) - rollout_corr/chi2_seq:np.float64(0.9885839202906936) - actor/pg_loss:np.float64(1.733703538775444e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.918888794236409e-05) - actor/ppo_kl:np.float64(-5.2708012820090744e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.047866626060567796) - perf/mfu/actor:np.float64(0.18982802399143323) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.7109375) - perf/cpu_memory_used_gb:np.float64(107.20830154418945) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.056533027440309525 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.056533027440309525 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1849.33203125 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.02734375 - response_length_non_aborted/mean:1849.33203125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.02734375 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.295259714126587e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2005517426878214) - timing_s/agent_loop/generate_sequences/max:np.float64(71.4071844201535) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.98496175607579) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(71.4071844201535) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:78.54274812154472 - timing_s/reward:0.12314253114163876 - timing_s/old_log_prob:4.655178176239133 - timing_s/adv:0.1377553455531597 - timing_s/update_actor:19.54215112142265 - timing_s/step:103.08848447538912 - timing_s/stop_profile:0.00012132711708545685 - timing_per_token_ms/gen:0.16590185248800712 - timing_per_token_ms/update_actor:0.03613690407285509 - timing_per_token_ms/adv:0.00025473407082194033 - perf/total_num_tokens:540781 - perf/time_per_step:103.08848447538912 - perf/throughput:655.7243065896264 (TaskRunner pid=2281424) Training Progress: 99%|█████████▉| 99/100 [2:23:03<01:45, 105.44s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] We are given the electric field $ E $ due to a point charge $ q $ at a distance $ r $, and we are asked to find the magnitude of the charge. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Plug in the known values (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Compute the square of the distance (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (0.50)^2 = 0.25 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Multiply by the electric field (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 2.0 \cdot 0.25 = 0.50 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 4: Divide by Coulomb's constant (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} \approx \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 5: Perform the division (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 6: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We know that $ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $, so: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 7: Choose the closest option (TaskRunner pid=2281424) (TaskRunner pid=2281424) Looking at the answer choices: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - A: 50 pC (TaskRunner pid=2281424) - B: 56 pC (TaskRunner pid=2281424) - C: 60 pC (TaskRunner pid=2281424) - D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest value is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $ at a distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ from a point charge. Using the formula $ E = k \frac{q}{r^2} $, we solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Converting to picocoulombs: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest answer is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2281424) step:100 - global_seqlen/min:33724 - global_seqlen/max:110235 - global_seqlen/minmax_diff:76511 - global_seqlen/balanced_min:67861 - global_seqlen/balanced_max:68027 - global_seqlen/mean:67950.125 - actor/entropy:0.45818907022476196 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5070421695709229 - training/rollout_probs_diff_mean:0.0032882497180253267 - training/rollout_probs_diff_std:0.007731531746685505 - training/rollout_actor_probs_pearson_corr:0.9995552897453308 - rollout_corr/rollout_is_mean:1.0000216960906982 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.213497959426604e-06 - rollout_corr/rollout_is_max:1.5396182537078857 - rollout_corr/rollout_is_min:0.46327918767929077 - rollout_corr/rollout_is_std:0.02762351557612419 - rollout_corr/rollout_is_eff_sample_size:0.9992375231521491 - rollout_corr/rollout_is_seq_mean:1.000037670135498 - rollout_corr/rollout_is_seq_std:0.0009224399691447616 - rollout_corr/rollout_is_seq_max:1.0046027898788452 - rollout_corr/rollout_is_seq_min:0.9973651766777039 - rollout_corr/rollout_is_seq_max_deviation:0.004602789878845215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4568524220958352) - rollout_corr/training_log_ppl:np.float64(0.35900981855229475) - rollout_corr/kl:np.float64(0.0003901403655550695) - rollout_corr/k3_kl:np.float64(0.00036932118109689327) - rollout_corr/rollout_ppl:np.float64(1.4562692898325622) - rollout_corr/rollout_log_ppl:np.float64(0.3586196781543549) - rollout_corr/log_ppl_diff:np.float64(0.000390140397939831) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008009577286429703) - rollout_corr/log_ppl_diff_max:np.float64(0.003635212779045105) - rollout_corr/log_ppl_diff_min:np.float64(-0.003899216651916504) - rollout_corr/ppl_ratio:np.float64(1.00039070751518) - rollout_corr/chi2_token:np.float64(0.0006887984927743673) - rollout_corr/chi2_seq:np.float64(3.438023897819221) - actor/pg_loss:np.float64(-1.3369019143283367e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.500477859892271e-05) - actor/ppo_kl:np.float64(6.788543978900052e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.04646363575011492) - perf/mfu/actor:np.float64(0.1938518522795187) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.7109375) - perf/cpu_memory_used_gb:np.float64(107.14533233642578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7296875) - val-aux/sciknoweval/reward/std@16:np.float64(0.21113028788197769) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.8195625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18312693662059415) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6398875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1769580686853267) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7299875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21049045089342963) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8962625000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1329636020160485) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5704125) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12181204475036858) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.76555) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18390036772261203) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.94765) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07747202634384016) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.52575) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07452105831546266) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7921125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15178369995448104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9732) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03523755515622687) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49838750000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.045061773818715276) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.8134375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11988946359651878) - val-aux/sciknoweval/score/mean@16:np.float64(0.7296875) - val-aux/sciknoweval/score/std@16:np.float64(0.21113028788197769) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.8195625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18312693662059415) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6398875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1769580686853267) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7299875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21049045089342963) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8962625000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1329636020160485) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5704125) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12181204475036858) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.76555) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18390036772261203) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.94765) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07747202634384016) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.52575) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07452105831546266) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7921125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15178369995448104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9732) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03523755515622687) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49838750000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.045061773818715276) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.8134375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11988946359651878) - val-core/sciknoweval/acc/mean@16:np.float64(0.7296875) - val-aux/sciknoweval/acc/std@16:np.float64(0.21113028788197769) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.8195625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18312693662059415) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6398875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1769580686853267) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7299875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21049045089342963) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8962625000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1329636020160485) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5704125) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12181204475036858) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.76555) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18390036772261203) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.94765) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07747202634384016) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.52575) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07452105831546266) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7921125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15178369995448104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9732) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03523755515622687) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49838750000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.045061773818715276) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.8134375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11988946359651878) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.91796875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.12921982287160344) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.969875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.07402285219911191) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8673375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.14334823308252115) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9178625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.1290617057662198) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99245) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.02834928965541194) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.80435) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.13896621816207683) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9458125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.10365239649604263) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9992375000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.006519985280372976) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7432874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.112524959047603) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9701000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.07166964518321907) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0010787124614257437) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.695975) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07060170833571836) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9865999999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03874947109834628) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023091811686754227 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023091811686754227 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1854.16015625 - response_length/max:8192.0 - response_length/min:336.0 - response_length/clip_ratio:0.01171875 - response_length_non_aborted/mean:1854.16015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:336.0 - response_length_non_aborted/clip_ratio:0.01171875 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.544752538204193e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.829353081062436) - timing_s/agent_loop/generate_sequences/max:np.float64(67.18781132809818) - timing_s/agent_loop/generate_sequences/mean:np.float64(17.904436235730827) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(67.18781132809818) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:74.55190750397742 - timing_s/reward:0.11913642473518848 - timing_s/old_log_prob:4.5896848775446415 - timing_s/adv:0.12158972769975662 - timing_s/update_actor:19.037666445598006 - timing_s/step:98.50110033527017 - timing_s/testing:211.57785162888467 - timing_s/save_checkpoint:12.600243583321571 - timing_s/stop_profile:0.00014124438166618347 - timing_per_token_ms/gen:0.15706215436987647 - timing_per_token_ms/update_actor:0.035021397027595616 - timing_per_token_ms/adv:0.000223674584299434 - perf/total_num_tokens:543601 - perf/time_per_step:98.50110033527017 - perf/throughput:689.8412786122876 (TaskRunner pid=2281424) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2281424) "np.float64(0.7296875), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2281424) "np.float64(0.21113028788197769), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2281424) "np.float64(0.8195625), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2281424) "np.float64(0.18312693662059415), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2281424) "np.float64(0.6398875), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2281424) "np.float64(0.1769580686853267), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2281424) "np.float64(0.7299875), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2281424) "np.float64(0.21049045089342963), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2281424) "np.float64(0.8962625000000001), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2281424) "np.float64(0.1329636020160485), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2281424) "np.float64(0.5704125), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2281424) "np.float64(0.12181204475036858), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2281424) "np.float64(0.76555), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2281424) "np.float64(0.18390036772261203), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2281424) "np.float64(0.94765), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2281424) "np.float64(0.07747202634384016), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2281424) "np.float64(0.52575), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2281424) "np.float64(0.07452105831546266), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2281424) "np.float64(0.7921125), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2281424) "np.float64(0.15178369995448104), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2281424) "np.float64(0.9732), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2281424) "np.float64(0.03523755515622687), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2281424) "np.float64(0.49838750000000004), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2281424) "np.float64(0.045061773818715276), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2281424) "np.float64(0.8134375), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2281424) "np.float64(0.11988946359651878), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2281424) "np.float64(0.7296875), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2281424) "np.float64(0.21113028788197769), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2281424) "np.float64(0.8195625), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2281424) "np.float64(0.18312693662059415), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2281424) "np.float64(0.6398875), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2281424) "np.float64(0.1769580686853267), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2281424) "np.float64(0.7299875), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2281424) "np.float64(0.21049045089342963), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2281424) "np.float64(0.8962625000000001), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2281424) "np.float64(0.1329636020160485), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2281424) "np.float64(0.5704125), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2281424) "np.float64(0.12181204475036858), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2281424) "np.float64(0.76555), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2281424) "np.float64(0.18390036772261203), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2281424) "np.float64(0.94765), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2281424) "np.float64(0.07747202634384016), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2281424) "np.float64(0.52575), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2281424) "np.float64(0.07452105831546266), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2281424) "np.float64(0.7921125), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2281424) "np.float64(0.15178369995448104), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2281424) "np.float64(0.9732), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2281424) "np.float64(0.03523755515622687), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2281424) "np.float64(0.49838750000000004), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2281424) "np.float64(0.045061773818715276), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2281424) "np.float64(0.8134375), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2281424) "np.float64(0.11988946359651878), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2281424) "np.float64(0.7296875), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2281424) "np.float64(0.21113028788197769), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2281424) "np.float64(0.8195625), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2281424) "np.float64(0.18312693662059415), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2281424) "np.float64(0.6398875), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2281424) "np.float64(0.1769580686853267), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2281424) "np.float64(0.7299875), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2281424) "np.float64(0.21049045089342963), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2281424) "np.float64(0.8962625000000001), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2281424) "np.float64(0.1329636020160485), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2281424) "np.float64(0.5704125), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2281424) "np.float64(0.12181204475036858), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2281424) "np.float64(0.76555), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2281424) "np.float64(0.18390036772261203), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2281424) "np.float64(0.94765), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2281424) "np.float64(0.07747202634384016), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2281424) "np.float64(0.52575), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2281424) "np.float64(0.07452105831546266), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2281424) "np.float64(0.7921125), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2281424) "np.float64(0.15178369995448104), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2281424) "np.float64(0.9732), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2281424) "np.float64(0.03523755515622687), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2281424) "np.float64(0.49838750000000004), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2281424) "np.float64(0.045061773818715276), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2281424) "np.float64(0.8134375), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2281424) 'np.float64(0.11988946359651878), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.91796875), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2281424) 'np.float64(0.12921982287160344), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.969875), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2281424) 'np.float64(0.07402285219911191), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(0.8673375), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2281424) 'np.float64(0.14334823308252115), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9178625), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2281424) 'np.float64(0.1290617057662198), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(0.99245), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2281424) 'np.float64(0.02834928965541194), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.80435), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2281424) 'np.float64(0.13896621816207683), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.9458125), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2281424) 'np.float64(0.10365239649604263), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@8/mean': " (TaskRunner pid=2281424) 'np.float64(0.9992375000000001), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@8/std': " (TaskRunner pid=2281424) 'np.float64(0.006519985280372976), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2281424) 'np.float64(0.7432874999999999), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2281424) 'np.float64(0.112524959047603), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2281424) 'np.float64(0.9701000000000001), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2281424) 'np.float64(0.07166964518321907), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@16/mean': " (TaskRunner pid=2281424) 'np.float64(0.9999500000000001), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@16/std': " (TaskRunner pid=2281424) 'np.float64(0.0010787124614257437), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(0.695975), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2281424) 'np.float64(0.07060170833571836), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2281424) 'np.float64(0.9865999999999999), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2281424) "np.float64(0.03874947109834628), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2281424) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2281424) 'np.float64(2.0)}') (TaskRunner pid=2281424) Training Progress: 100%|██████████| 100/100 [2:28:26<00:00, 170.63s/it] Training Progress: 100%|██████████| 100/100 [2:28:26<00:00, 89.06s/it] (TaskRunner pid=2281424) wandb: updating run metadata (TaskRunner pid=2281424) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2281424) wandb: uploading output.log (TaskRunner pid=2281424) wandb: uploading history steps 98-99, summary, console lines 1063-1188 (TaskRunner pid=2281424) wandb: (TaskRunner pid=2281424) wandb: Run history: (TaskRunner pid=2281424) wandb: actor/entropy ▂▂▂▂▂▁▂▃▂▃▄▃▄▃▂▃▄▅▄▄▅▄▅▅▅▅▆▅▆▅▆▄▇█▆▅▆▇▇▆ (TaskRunner pid=2281424) wandb: actor/grad_norm ▄▄▄▅▅▅▄█▆▃▃▃▄▄▅▃▃▃▃▃▃▃▄▃▄▂▂▂▂▁▁▃▂▂▂▂▁▁▂▁ (TaskRunner pid=2281424) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2281424) wandb: actor/lr ▁▂▅▅▆███████████████████████████████████ (TaskRunner pid=2281424) wandb: actor/pg_clipfrac ▂▄█▄▆▃█▄▄▅▄▇▆▃▄▃▄▄▂▄▂▃▃▅▂▂▂▃▂▂▂▁▂▁▂▂▂▂▂▁ (TaskRunner pid=2281424) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▂▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▂ (TaskRunner pid=2281424) wandb: actor/pg_loss ▁▅▃▅▄▄▄▇▃▁▅▄▁█▃▂▄▄▅▄▃▃▅▄▅▄▅▂▃▅▄▅▅▄▄▅▃▁▄▄ (TaskRunner pid=2281424) wandb: actor/ppo_kl ▃▂▄▅▅▇▁▆█▅▄▄▅▃▄▄▃▅▆▃▃▂▄▄▄▄▃▄▄▃▂▄▃▃▄▄▄▄▄▄ (TaskRunner pid=2281424) wandb: critic/advantages/max ███▁████████████████████████████████████ (TaskRunner pid=2281424) wandb: critic/advantages/mean ▇▆▇▆▇█▇█▄▇▇▄▇▇▆██▇▅▅▂█▅▇▄▁▅▅▆▄▆▇▄▅▇▆▆▆▅▅ (TaskRunner pid=2281424) wandb: +204 ... (TaskRunner pid=2281424) wandb: (TaskRunner pid=2281424) wandb: Run summary: (TaskRunner pid=2281424) wandb: actor/entropy 0.45819 (TaskRunner pid=2281424) wandb: actor/grad_norm 0.04646 (TaskRunner pid=2281424) wandb: actor/kl_loss 0 (TaskRunner pid=2281424) wandb: actor/lr 0.0 (TaskRunner pid=2281424) wandb: actor/pg_clipfrac 5e-05 (TaskRunner pid=2281424) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2281424) wandb: actor/pg_loss -1e-05 (TaskRunner pid=2281424) wandb: actor/ppo_kl 7e-05 (TaskRunner pid=2281424) wandb: critic/advantages/max 0.875 (TaskRunner pid=2281424) wandb: critic/advantages/mean -0.02309 (TaskRunner pid=2281424) wandb: +204 ... (TaskRunner pid=2281424) wandb: (TaskRunner pid=2281424) wandb: 🚀 View run qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/6ig3l55l (TaskRunner pid=2281424) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2281424) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2281424) wandb: Find logs at: ./wandb/run-20260703_032040-6ig3l55l/logs (TaskRunner pid=2281424) Exception ignored in atexit callback: .teardown_atexit at 0x7d739de7b920> (TaskRunner pid=2281424) Traceback (most recent call last): (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2281424) conn.teardown(hooks.exit_code) (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2281424) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2281424) return future.result() (TaskRunner pid=2281424) ^^^^^^^^^^^^^^^ (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2281424) return self.__get_result() (TaskRunner pid=2281424) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2281424) raise self._exception (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2281424) return await fn() (TaskRunner pid=2281424) ^^^^^^^^^^ (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2281424) await self._client.close() (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2281424) await self._writer.wait_closed() (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2281424) await self._protocol._get_close_waiter(self) (TaskRunner pid=2281424) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 05:49:13] Finished Qwen/Qwen3-8B physics grpo. [2026-07-03 05:49:13] Starting Qwen/Qwen3-8B physics rlsd_tr. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B 2026-07-03 05:49:15,332 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 05:49:25,205 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2320396) TaskRunner hostname: mole-workspace-rw, PID: 2320396 (TaskRunner pid=2320396) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2320396) 'calculate_entropy': False, (TaskRunner pid=2320396) 'calculate_sum_pi_squared': False, (TaskRunner pid=2320396) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2320396) 'async_save': False, (TaskRunner pid=2320396) 'load_contents': ['model', (TaskRunner pid=2320396) 'optimizer', (TaskRunner pid=2320396) 'extra'], (TaskRunner pid=2320396) 'save_contents': ['model', (TaskRunner pid=2320396) 'optimizer', (TaskRunner pid=2320396) 'extra']}, (TaskRunner pid=2320396) 'clip_ratio': 0.2, (TaskRunner pid=2320396) 'clip_ratio_c': 3.0, (TaskRunner pid=2320396) 'clip_ratio_high': 0.28, (TaskRunner pid=2320396) 'clip_ratio_low': 0.2, (TaskRunner pid=2320396) 'data_loader_seed': 42, (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_coeff': 0, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'freeze_vision_tower': False, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'forward_only': False, (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'full_determinism': False, (TaskRunner pid=2320396) 'model_dtype': 'fp32', (TaskRunner pid=2320396) 'offload_policy': False, (TaskRunner pid=2320396) 'optimizer_offload': False, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'seed': 42, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_orig_params': False, (TaskRunner pid=2320396) 'use_torch_compile': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'grad_clip': 1.0, (TaskRunner pid=2320396) 'kl_loss_coef': 0.001, (TaskRunner pid=2320396) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2320396) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2320396) 'loss_scale_factor': None, (TaskRunner pid=2320396) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2320396) 'betas': [0.9, 0.999], (TaskRunner pid=2320396) 'clip_grad': 1.0, (TaskRunner pid=2320396) 'lr': 1e-06, (TaskRunner pid=2320396) 'lr_scheduler_type': 'constant', (TaskRunner pid=2320396) 'lr_warmup_steps': 10, (TaskRunner pid=2320396) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2320396) 'min_lr_ratio': 0.0, (TaskRunner pid=2320396) 'num_cycles': 0.5, (TaskRunner pid=2320396) 'optimizer': 'AdamW', (TaskRunner pid=2320396) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2320396) 'override_optimizer_config': None, (TaskRunner pid=2320396) 'total_training_steps': -1, (TaskRunner pid=2320396) 'warmup_style': None, (TaskRunner pid=2320396) 'weight_decay': 0.01}, (TaskRunner pid=2320396) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2320396) 'clip_cov_lb': 1.0, (TaskRunner pid=2320396) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2320396) 'clip_cov_ub': 5.0, (TaskRunner pid=2320396) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2320396) 'loss_mode': 'rlsd', (TaskRunner pid=2320396) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2320396) 'ppo_epochs': 1, (TaskRunner pid=2320396) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2320396) 'ppo_micro_batch_size': None, (TaskRunner pid=2320396) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2320396) 'ppo_mini_batch_size': 8, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'rollout_n': 8, (TaskRunner pid=2320396) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2320396) 'mode': 'disabled', (TaskRunner pid=2320396) 'record_file': None, (TaskRunner pid=2320396) 'replay_file': None}, (TaskRunner pid=2320396) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2320396) 'alpha': 0.5, (TaskRunner pid=2320396) 'distillation_add_tail': True, (TaskRunner pid=2320396) 'distillation_topk': 100, (TaskRunner pid=2320396) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2320396) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2320396) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'loss_weight': 0.0, (TaskRunner pid=2320396) 'mask': 'all'}, (TaskRunner pid=2320396) 'feedback_template': '\n' (TaskRunner pid=2320396) 'The ' (TaskRunner pid=2320396) 'following ' (TaskRunner pid=2320396) 'is ' (TaskRunner pid=2320396) 'feedback ' (TaskRunner pid=2320396) 'from ' (TaskRunner pid=2320396) 'your ' (TaskRunner pid=2320396) 'unsuccessful ' (TaskRunner pid=2320396) 'earlier ' (TaskRunner pid=2320396) 'attempt:\n' (TaskRunner pid=2320396) '\n' (TaskRunner pid=2320396) '{feedback_raw}', (TaskRunner pid=2320396) 'full_logit_distillation': True, (TaskRunner pid=2320396) 'include_environment_feedback': True, (TaskRunner pid=2320396) 'is_clip': 2, (TaskRunner pid=2320396) 'max_reprompt_len': 10240, (TaskRunner pid=2320396) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2320396) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2320396) '\n' (TaskRunner pid=2320396) 'Correctly ' (TaskRunner pid=2320396) 'solve ' (TaskRunner pid=2320396) 'the ' (TaskRunner pid=2320396) 'original ' (TaskRunner pid=2320396) 'question.', (TaskRunner pid=2320396) 'reprompt_truncation': 'right', (TaskRunner pid=2320396) 'solution_template': '\n' (TaskRunner pid=2320396) 'Correct ' (TaskRunner pid=2320396) 'solution:\n' (TaskRunner pid=2320396) '\n' (TaskRunner pid=2320396) '{successful_previous_attempt}', (TaskRunner pid=2320396) 'srpo_dynamic_weighting': False, (TaskRunner pid=2320396) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2320396) 'success_reward_threshold': 0.5, (TaskRunner pid=2320396) 'teacher_regularization': 'trust-region', (TaskRunner pid=2320396) 'teacher_update_rate': 0.1, (TaskRunner pid=2320396) 'token_reweight_decay_steps': 0, (TaskRunner pid=2320396) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2320396) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2320396) 'shuffle': False, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2320396) 'tau_neg': 1.05, (TaskRunner pid=2320396) 'tau_pos': 1.0, (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_dynamic_bsz': False, (TaskRunner pid=2320396) 'use_fused_kernels': False, (TaskRunner pid=2320396) 'use_kl_loss': False, (TaskRunner pid=2320396) 'use_prefix_grouper': False, (TaskRunner pid=2320396) 'use_remove_padding': True, (TaskRunner pid=2320396) 'use_torch_compile': True}, (TaskRunner pid=2320396) 'hybrid_engine': True, (TaskRunner pid=2320396) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2320396) 'custom_chat_template': None, (TaskRunner pid=2320396) 'enable_activation_offload': False, (TaskRunner pid=2320396) 'enable_gradient_checkpointing': True, (TaskRunner pid=2320396) 'exclude_modules': None, (TaskRunner pid=2320396) 'external_lib': None, (TaskRunner pid=2320396) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2320396) 'hf_config_path': None, (TaskRunner pid=2320396) 'lora_adapter_path': None, (TaskRunner pid=2320396) 'lora_alpha': 16, (TaskRunner pid=2320396) 'lora_rank': 0, (TaskRunner pid=2320396) 'override_config': {}, (TaskRunner pid=2320396) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'target_modules': 'all-linear', (TaskRunner pid=2320396) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2320396) 'num_shards': 4}, (TaskRunner pid=2320396) 'tokenizer_path': None, (TaskRunner pid=2320396) 'trust_remote_code': True, (TaskRunner pid=2320396) 'use_fused_kernels': False, (TaskRunner pid=2320396) 'use_liger': False, (TaskRunner pid=2320396) 'use_remove_padding': True, (TaskRunner pid=2320396) 'use_shm': False}, (TaskRunner pid=2320396) 'nccl_timeout': 600, (TaskRunner pid=2320396) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'forward_only': True, (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'full_determinism': False, (TaskRunner pid=2320396) 'model_dtype': 'fp32', (TaskRunner pid=2320396) 'offload_policy': False, (TaskRunner pid=2320396) 'optimizer_offload': False, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'seed': 42, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_orig_params': False, (TaskRunner pid=2320396) 'use_torch_compile': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2320396) 'log_prob_micro_batch_size': None, (TaskRunner pid=2320396) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2320396) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'rollout_n': 8, (TaskRunner pid=2320396) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2320396) 'mode': 'disabled', (TaskRunner pid=2320396) 'record_file': None, (TaskRunner pid=2320396) 'replay_file': None}, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_torch_compile': True}, (TaskRunner pid=2320396) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2320396) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2320396) 'agent_loop_config_path': None, (TaskRunner pid=2320396) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2320396) 'name': None, (TaskRunner pid=2320396) 'path': None}, (TaskRunner pid=2320396) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2320396) 'num_workers': 8}, (TaskRunner pid=2320396) 'calculate_log_probs': True, (TaskRunner pid=2320396) 'cudagraph_capture_sizes': None, (TaskRunner pid=2320396) 'data_parallel_size': 1, (TaskRunner pid=2320396) 'disable_log_stats': True, (TaskRunner pid=2320396) 'do_sample': True, (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'enable_chunked_prefill': True, (TaskRunner pid=2320396) 'enable_prefix_caching': True, (TaskRunner pid=2320396) 'enable_rollout_routing_replay': False, (TaskRunner pid=2320396) 'enforce_eager': False, (TaskRunner pid=2320396) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2320396) 'expert_parallel_size': 1, (TaskRunner pid=2320396) 'free_cache_engine': True, (TaskRunner pid=2320396) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2320396) 'ignore_eos': False, (TaskRunner pid=2320396) 'layered_summon': False, (TaskRunner pid=2320396) 'load_format': 'dummy', (TaskRunner pid=2320396) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2320396) 'log_prob_micro_batch_size': None, (TaskRunner pid=2320396) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2320396) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2320396) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2320396) 'max_model_len': 10240, (TaskRunner pid=2320396) 'max_num_batched_tokens': 10240, (TaskRunner pid=2320396) 'max_num_seqs': 1024, (TaskRunner pid=2320396) 'mode': 'async', (TaskRunner pid=2320396) 'multi_stage_wake_up': False, (TaskRunner pid=2320396) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'format': 'hermes', (TaskRunner pid=2320396) 'interaction_config_path': None, (TaskRunner pid=2320396) 'max_assistant_turns': None, (TaskRunner pid=2320396) 'max_parallel_calls': 1, (TaskRunner pid=2320396) 'max_tool_response_length': 256, (TaskRunner pid=2320396) 'max_user_turns': None, (TaskRunner pid=2320396) 'num_repeat_rollouts': None, (TaskRunner pid=2320396) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2320396) 'tool_config_path': None, (TaskRunner pid=2320396) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2320396) 'use_inference_chat_template': False}, (TaskRunner pid=2320396) 'n': 8, (TaskRunner pid=2320396) 'name': 'vllm', (TaskRunner pid=2320396) 'over_sample_rate': 0, (TaskRunner pid=2320396) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2320396) 'port': 9090, (TaskRunner pid=2320396) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2320396) 'prompt_length': 2048, (TaskRunner pid=2320396) 'quantization': None, (TaskRunner pid=2320396) 'quantization_config_file': None, (TaskRunner pid=2320396) 'response_length': 8192, (TaskRunner pid=2320396) 'scheduling_policy': 'fcfs', (TaskRunner pid=2320396) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2320396) 'skip_rollout': False, (TaskRunner pid=2320396) 'skip_tokenizer_init': True, (TaskRunner pid=2320396) 'temperature': 1.0, (TaskRunner pid=2320396) 'tensor_model_parallel_size': 2, (TaskRunner pid=2320396) 'top_k': -1, (TaskRunner pid=2320396) 'top_p': 1.0, (TaskRunner pid=2320396) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2320396) 'backend': None, (TaskRunner pid=2320396) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2320396) 'token2text': False}, (TaskRunner pid=2320396) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2320396) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2320396) 'do_sample': True, (TaskRunner pid=2320396) 'n': 16, (TaskRunner pid=2320396) 'temperature': 0.6, (TaskRunner pid=2320396) 'top_k': -1, (TaskRunner pid=2320396) 'top_p': 0.95}}}, (TaskRunner pid=2320396) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2320396) 'adv_estimator': 'grpo', (TaskRunner pid=2320396) 'gamma': 1.0, (TaskRunner pid=2320396) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2320396) 'horizon': 10000, (TaskRunner pid=2320396) 'kl_coef': 0.001, (TaskRunner pid=2320396) 'target_kl': 0.1, (TaskRunner pid=2320396) 'type': 'fixed'}, (TaskRunner pid=2320396) 'kl_penalty': 'kl', (TaskRunner pid=2320396) 'lam': 1.0, (TaskRunner pid=2320396) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2320396) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2320396) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2320396) 'loss_type': 'ppo_clip', (TaskRunner pid=2320396) 'rollout_is': 'token', (TaskRunner pid=2320396) 'rollout_is_batch_normalize': False, (TaskRunner pid=2320396) 'rollout_is_threshold': 2.0, (TaskRunner pid=2320396) 'rollout_rs': None, (TaskRunner pid=2320396) 'rollout_rs_threshold': None}, (TaskRunner pid=2320396) 'use_kl_in_reward': False, (TaskRunner pid=2320396) 'use_pf_ppo': False}, (TaskRunner pid=2320396) 'critic': (TaskRunner pid=2320396) {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2320396) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2320396) 'async_save': False, (TaskRunner pid=2320396) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2320396) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2320396) 'cliprange_value': 0.5, (TaskRunner pid=2320396) 'data_loader_seed': 42, (TaskRunner pid=2320396) 'enable': None, (TaskRunner pid=2320396) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2320396) 'forward_micro_batch_size': None, (TaskRunner pid=2320396) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2320396) 'grad_clip': 1.0, (TaskRunner pid=2320396) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2320396) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2320396) 'enable_activation_offload': False, (TaskRunner pid=2320396) 'enable_gradient_checkpointing': True, (TaskRunner pid=2320396) 'external_lib': None, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'forward_only': False, (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'full_determinism': False, (TaskRunner pid=2320396) 'model_dtype': 'fp32', (TaskRunner pid=2320396) 'offload_policy': False, (TaskRunner pid=2320396) 'optimizer_offload': False, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'seed': 42, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_orig_params': False, (TaskRunner pid=2320396) 'use_torch_compile': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'lora_alpha': 16, (TaskRunner pid=2320396) 'lora_rank': 0, (TaskRunner pid=2320396) 'override_config': {}, (TaskRunner pid=2320396) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'target_modules': 'all-linear', (TaskRunner pid=2320396) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2320396) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'trust_remote_code': True, (TaskRunner pid=2320396) 'use_remove_padding': False, (TaskRunner pid=2320396) 'use_shm': False}, (TaskRunner pid=2320396) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2320396) 'betas': [0.9, 0.999], (TaskRunner pid=2320396) 'clip_grad': 1.0, (TaskRunner pid=2320396) 'lr': 1e-05, (TaskRunner pid=2320396) 'lr_scheduler_type': 'constant', (TaskRunner pid=2320396) 'lr_warmup_steps': -1, (TaskRunner pid=2320396) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2320396) 'min_lr_ratio': 0.0, (TaskRunner pid=2320396) 'num_cycles': 0.5, (TaskRunner pid=2320396) 'optimizer': 'AdamW', (TaskRunner pid=2320396) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2320396) 'override_optimizer_config': None, (TaskRunner pid=2320396) 'total_training_steps': -1, (TaskRunner pid=2320396) 'warmup_style': None, (TaskRunner pid=2320396) 'weight_decay': 0.01}, (TaskRunner pid=2320396) 'ppo_epochs': 1, (TaskRunner pid=2320396) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2320396) 'ppo_micro_batch_size': None, (TaskRunner pid=2320396) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2320396) 'ppo_mini_batch_size': 8, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'rollout_n': 8, (TaskRunner pid=2320396) 'shuffle': False, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_dynamic_bsz': False}, (TaskRunner pid=2320396) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2320396) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2320396) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2320396) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2320396) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2320396) 'dataloader_num_workers': 8, (TaskRunner pid=2320396) 'filter_overlong_prompts': True, (TaskRunner pid=2320396) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2320396) 'image_key': 'images', (TaskRunner pid=2320396) 'image_patch_size': 14, (TaskRunner pid=2320396) 'max_prompt_length': 2048, (TaskRunner pid=2320396) 'max_response_length': 8192, (TaskRunner pid=2320396) 'prompt_key': 'prompt', (TaskRunner pid=2320396) 'return_full_prompt': False, (TaskRunner pid=2320396) 'return_multi_modal_inputs': True, (TaskRunner pid=2320396) 'return_raw_chat': True, (TaskRunner pid=2320396) 'return_raw_input_ids': False, (TaskRunner pid=2320396) 'reward_fn_key': 'data_source', (TaskRunner pid=2320396) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2320396) 'seed': None, (TaskRunner pid=2320396) 'shuffle': True, (TaskRunner pid=2320396) 'tokenizer': None, (TaskRunner pid=2320396) 'tool_config_path': None, (TaskRunner pid=2320396) 'train_batch_size': 32, (TaskRunner pid=2320396) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2320396) 'train_max_samples': 3200, (TaskRunner pid=2320396) 'truncation': 'error', (TaskRunner pid=2320396) 'trust_remote_code': True, (TaskRunner pid=2320396) 'use_shm': False, (TaskRunner pid=2320396) 'val_batch_size': None, (TaskRunner pid=2320396) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2320396) 'val_max_samples': -1, (TaskRunner pid=2320396) 'validation_shuffle': False, (TaskRunner pid=2320396) 'video_key': 'videos'}, (TaskRunner pid=2320396) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2320396) 'cuda-memory-usage': 'true', (TaskRunner pid=2320396) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2320396) 'capture-range-end': None, (TaskRunner pid=2320396) 'cuda-graph-trace': 'graph', (TaskRunner pid=2320396) 'cuda-memory-usage': 'true', (TaskRunner pid=2320396) 'kill': 'none', (TaskRunner pid=2320396) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2320396) 'torch_memory': {'context': 'all', (TaskRunner pid=2320396) 'kw_args': {}, (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'stacks': 'all', (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2320396) 'profile_continuous_steps': False, (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'steps': None, (TaskRunner pid=2320396) 'tool': None}, (TaskRunner pid=2320396) 'max_model_len': 10240, (TaskRunner pid=2320396) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B', (TaskRunner pid=2320396) 'include_dashboard': False, (TaskRunner pid=2320396) 'num_cpus': None, (TaskRunner pid=2320396) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2320396) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2320396) 'USER': 'root'}}}, (TaskRunner pid=2320396) 'timeline_json_file': None}, (TaskRunner pid=2320396) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2320396) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2320396) 'name': 'custom_reward_manager', (TaskRunner pid=2320396) 'path': None}, (TaskRunner pid=2320396) 'name': 'naive', (TaskRunner pid=2320396) 'source': 'register'}, (TaskRunner pid=2320396) 'reward_model': {'enable': False, (TaskRunner pid=2320396) 'enable_resource_pool': False, (TaskRunner pid=2320396) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2320396) 'launch_reward_fn_async': False, (TaskRunner pid=2320396) 'max_length': None, (TaskRunner pid=2320396) 'micro_batch_size': None, (TaskRunner pid=2320396) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2320396) 'model': {'external_lib': None, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'override_config': {}, (TaskRunner pid=2320396) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2320396) 'trust_remote_code': False, (TaskRunner pid=2320396) 'use_fused_kernels': False, (TaskRunner pid=2320396) 'use_remove_padding': False, (TaskRunner pid=2320396) 'use_shm': False}, (TaskRunner pid=2320396) 'n_gpus_per_node': 8, (TaskRunner pid=2320396) 'nnodes': 0, (TaskRunner pid=2320396) 'num_workers': 1, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'reward_loop_class_name': None, (TaskRunner pid=2320396) 'reward_loop_module_path': None, (TaskRunner pid=2320396) 'reward_loop_source': 'register', (TaskRunner pid=2320396) 'reward_manager': 'naive', (TaskRunner pid=2320396) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2320396) 'cudagraph_capture_sizes': None, (TaskRunner pid=2320396) 'data_parallel_size': 1, (TaskRunner pid=2320396) 'disable_log_stats': True, (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'enable_chunked_prefill': True, (TaskRunner pid=2320396) 'enable_prefix_caching': True, (TaskRunner pid=2320396) 'enforce_eager': True, (TaskRunner pid=2320396) 'engine_kwargs': {}, (TaskRunner pid=2320396) 'expert_parallel_size': 1, (TaskRunner pid=2320396) 'free_cache_engine': True, (TaskRunner pid=2320396) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2320396) 'limit_images': None, (TaskRunner pid=2320396) 'load_format': 'auto', (TaskRunner pid=2320396) 'max_model_len': None, (TaskRunner pid=2320396) 'max_num_batched_tokens': 8192, (TaskRunner pid=2320396) 'max_num_seqs': 1024, (TaskRunner pid=2320396) 'name': '???', (TaskRunner pid=2320396) 'prompt_length': 2048, (TaskRunner pid=2320396) 'response_length': 2048, (TaskRunner pid=2320396) 'skip_tokenizer_init': False, (TaskRunner pid=2320396) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2320396) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2320396) 'memory_limit_mb': 1024, (TaskRunner pid=2320396) 'url': None}, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_dynamic_bsz': False, (TaskRunner pid=2320396) 'use_reward_loop': False}, (TaskRunner pid=2320396) 'trainer': {'balance_batch': True, (TaskRunner pid=2320396) 'critic_warmup': 0, (TaskRunner pid=2320396) 'default_hdfs_dir': None, (TaskRunner pid=2320396) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2320396) 'del_local_ckpt_after_load': False, (TaskRunner pid=2320396) 'device': 'cuda', (TaskRunner pid=2320396) 'esi_redundant_time': 0, (TaskRunner pid=2320396) 'experiment_name': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2320396) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2320396) 'log_val_generations': 0, (TaskRunner pid=2320396) 'logger': ['console', 'wandb'], (TaskRunner pid=2320396) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2320396) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2320396) 'n_gpus_per_node': 8, (TaskRunner pid=2320396) 'nnodes': 1, (TaskRunner pid=2320396) 'project_name': 'SDPO-root', (TaskRunner pid=2320396) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2320396) 'resume_from_path': None, (TaskRunner pid=2320396) 'resume_mode': 'auto', (TaskRunner pid=2320396) 'rollout_data_dir': None, (TaskRunner pid=2320396) 'save_freq': 10, (TaskRunner pid=2320396) 'test_freq': 10, (TaskRunner pid=2320396) 'total_epochs': 30, (TaskRunner pid=2320396) 'total_training_steps': 100, (TaskRunner pid=2320396) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2320396) 'val_before_train': False, (TaskRunner pid=2320396) 'val_only': False, (TaskRunner pid=2320396) 'validation_data_dir': None}, (TaskRunner pid=2320396) 'transfer_queue': {'enable': False}, (TaskRunner pid=2320396) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2320396) 'dir': '/users/root/SDPO', (TaskRunner pid=2320396) 'log_dir': '/users/root/output', (TaskRunner pid=2320396) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2320396) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2320396) use_critic=need_critic(config), (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2320396) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2320396) Using dataset class: RLHFDataset (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2320396) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2320396) dataset len: 720 (TaskRunner pid=2320396) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2320396) WARNING:2026-07-03 05:49:34,055:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2320396) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2320396) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (WorkerDict pid=2320775) [W703 05:49:43.667210084 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:35715 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2320772) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2320775) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2320775) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2320779) [W703 05:49:43.905871872 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:35715 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2320775) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2320775) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2320775) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2320775) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2320772) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2320773) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2320773) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2320774) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.91s/it] [repeated 15x across cluster] (WorkerDict pid=2320774) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.62s/it] [repeated 7x across cluster] (WorkerDict pid=2320772) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2320772) Actor use_remove_padding=True (WorkerDict pid=2320772) Actor use_fused_kernels=False (WorkerDict pid=2320772) Actor use_prefix_grouper=False (WorkerDict pid=2320774) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2320774) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2320774) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2320774) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2320773) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2320772) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2320772) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2320778) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2320778) warnings.warn( (WorkerDict pid=2320772) reference model: Qwen/Qwen3-8B (WorkerDict pid=2320778) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2320778) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2320772) Model config after override: Qwen3Config { (WorkerDict pid=2320772) "architectures": [ (WorkerDict pid=2320772) "Qwen3ForCausalLM" (WorkerDict pid=2320772) ], (WorkerDict pid=2320772) "attention_bias": false, (WorkerDict pid=2320772) "attention_dropout": 0.0, (WorkerDict pid=2320772) "dtype": "bfloat16", (WorkerDict pid=2320772) "eos_token_id": 151645, (WorkerDict pid=2320772) "head_dim": 128, (WorkerDict pid=2320772) "hidden_act": "silu", (WorkerDict pid=2320772) "hidden_size": 4096, (WorkerDict pid=2320772) "initializer_range": 0.02, (WorkerDict pid=2320772) "intermediate_size": 12288, (WorkerDict pid=2320772) "layer_types": [ (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention" (WorkerDict pid=2320772) ], (WorkerDict pid=2320772) "max_position_embeddings": 40960, (WorkerDict pid=2320772) "max_window_layers": 36, (WorkerDict pid=2320772) "model_type": "qwen3", (WorkerDict pid=2320772) "num_attention_heads": 32, (WorkerDict pid=2320772) "num_hidden_layers": 36, (WorkerDict pid=2320772) "num_key_value_heads": 8, (WorkerDict pid=2320772) "pad_token_id": 151643, (WorkerDict pid=2320772) "rms_norm_eps": 1e-06, (WorkerDict pid=2320772) "rope_scaling": null, (WorkerDict pid=2320772) "rope_theta": 1000000, (WorkerDict pid=2320772) "sliding_window": null, (WorkerDict pid=2320772) "tie_word_embeddings": false, (WorkerDict pid=2320772) "transformers_version": "4.57.1", (WorkerDict pid=2320772) "use_cache": true, (WorkerDict pid=2320772) "use_sliding_window": false, (WorkerDict pid=2320772) "vocab_size": 151936 (WorkerDict pid=2320772) } (WorkerDict pid=2320772) (WorkerDict pid=2320776) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2320774) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2320772) Ref use_remove_padding=True (WorkerDict pid=2320772) Ref use_fused_kernels=False (WorkerDict pid=2320772) Ref use_prefix_grouper=False (WorkerDict pid=2320774) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2320774) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2320396) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2320775) Loading checkpoint shards: 80%|████████ | 4/5 [00:11<00:02, 2.86s/it] [repeated 15x across cluster] (WorkerDict pid=2320775) Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.18s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.58s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2321709) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2321709) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2321709) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2321709) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2321706) ['serve', (vLLMHttpServer pid=2321706) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2321706) '--dtype', (vLLMHttpServer pid=2321706) 'bfloat16', (vLLMHttpServer pid=2321706) '--load_format', (vLLMHttpServer pid=2321706) 'dummy', (vLLMHttpServer pid=2321706) '--trust_remote_code', (vLLMHttpServer pid=2321706) '--max_model_len', (vLLMHttpServer pid=2321706) '40960', (vLLMHttpServer pid=2321706) '--max_num_seqs', (vLLMHttpServer pid=2321706) '1024', (vLLMHttpServer pid=2321706) '--enable_chunked_prefill', (vLLMHttpServer pid=2321706) '--max_num_batched_tokens', (vLLMHttpServer pid=2321706) '10240', (vLLMHttpServer pid=2321706) '--enable_prefix_caching', (vLLMHttpServer pid=2321706) '--enable_sleep_mode', (vLLMHttpServer pid=2321706) '--logprobs_mode', (vLLMHttpServer pid=2321706) 'processed_logprobs', (vLLMHttpServer pid=2321706) '--disable_custom_all_reduce', (vLLMHttpServer pid=2321706) '--gpu_memory_utilization', (vLLMHttpServer pid=2321706) '0.8', (vLLMHttpServer pid=2321706) '--disable_log_stats', (vLLMHttpServer pid=2321706) '--tensor_parallel_size', (vLLMHttpServer pid=2321706) '2', (vLLMHttpServer pid=2321706) '--seed', (vLLMHttpServer pid=2321706) '0', (vLLMHttpServer pid=2321706) '--override_generation_config', (vLLMHttpServer pid=2321706) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2321706) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2321706) '--hf_overrides', (vLLMHttpServer pid=2321706) '{}', (vLLMHttpServer pid=2321706) '--scheduling_policy', (vLLMHttpServer pid=2321706) 'fcfs'] (vLLMHttpServer pid=2321709) WARNING 07-03 05:50:57 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2320778) WARNING 07-03 05:51:05 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2321708) WARNING 07-03 05:50:57 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2320774) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/aj/faj75csb7ofozv5in77ssqc762na7cdfwj3wkvfkkmz6awioro7y/.2320778.137164054263488.tmp' (vLLMHttpServer pid=2321708) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2321708) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2321708) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2321708) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2320776) 2026-07-03 05:51:18,431 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2320776) 2026-07-03 05:51:18,445 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2320772) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are to find the magnitude of the charge $ q $ using the formula: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rearranging to solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the given values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2320396) step:10 - global_seqlen/min:16491 - global_seqlen/max:30025 - global_seqlen/minmax_diff:13534 - global_seqlen/balanced_min:25337 - global_seqlen/balanced_max:25538 - global_seqlen/mean:25375.0 - actor/entropy:0.31971633434295654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2820093035697937 - training/rollout_probs_diff_mean:0.0033090896904468536 - training/rollout_probs_diff_std:0.009006068110466003 - training/rollout_actor_probs_pearson_corr:0.9993484020233154 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:0.9999304413795471 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7673910856246948 - rollout_corr/rollout_is_min:0.5399474501609802 - rollout_corr/rollout_is_std:0.02919088862836361 - rollout_corr/rollout_is_eff_sample_size:0.9991483794554346 - rollout_corr/rollout_is_seq_mean:0.9999055862426758 - rollout_corr/rollout_is_seq_std:0.0017869374714791775 - rollout_corr/rollout_is_seq_max:1.0064821243286133 - rollout_corr/rollout_is_seq_min:0.9924333095550537 - rollout_corr/rollout_is_seq_max_deviation:0.007566690444946289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36662180069834) - rollout_corr/training_log_ppl:np.float64(0.2933628953032894) - rollout_corr/kl:np.float64(0.0005606463353959867) - rollout_corr/k3_kl:np.float64(0.0005264203514343535) - rollout_corr/rollout_ppl:np.float64(1.3658109386451542) - rollout_corr/rollout_log_ppl:np.float64(0.2928022450651042) - rollout_corr/log_ppl_diff:np.float64(0.0005606502381851897) - rollout_corr/log_ppl_abs_diff:np.float64(0.001444778943550773) - rollout_corr/log_ppl_diff_max:np.float64(0.009412601590156555) - rollout_corr/log_ppl_diff_min:np.float64(-0.004387408494949341) - rollout_corr/ppl_ratio:np.float64(1.0005626599304378) - rollout_corr/chi2_token:np.float64(0.0009579714387655258) - rollout_corr/chi2_seq:np.float64(0.6561292943079025) - actor/pg_loss:np.float64(0.0034951333655044436) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002724248687400177) - actor/ppo_kl:np.float64(6.655702073254943e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(2924.901734317187) - self_distillation/token_reweight_w_std:np.float64(77176.66680757602) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011562679428607) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08802842449222226) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.22159437276422977) - perf/mfu/actor:np.float64(0.058286523178669794) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(191.5031852722168) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.58671875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2080057629527512) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6764374999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1762086184055891) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.500725) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17562203990145614) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5888625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20769851667696293) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7466875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12697903624119594) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4284625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12684135690194748) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6092875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1746588055788758) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7941499999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0796429418879614) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3798125) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07572344702548002) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6249750000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13772658944496724) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8243375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04642849444514059) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3520625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03889823328050272) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6363624999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10581089168215327) - val-aux/sciknoweval/score/mean@16:np.float64(0.58671875) - val-aux/sciknoweval/score/std@16:np.float64(0.2080057629527512) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6764374999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1762086184055891) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.500725) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17562203990145614) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5888625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20769851667696293) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7466875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12697903624119594) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4284625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12684135690194748) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6092875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1746588055788758) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7941499999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0796429418879614) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3798125) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07572344702548002) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6249750000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13772658944496724) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8243375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04642849444514059) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3520625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03889823328050272) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6363624999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10581089168215327) - val-core/sciknoweval/acc/mean@16:np.float64(0.58671875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2080057629527512) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6764374999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1762086184055891) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.500725) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17562203990145614) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5888625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20769851667696293) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7466875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12697903624119594) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4284625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12684135690194748) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6092875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1746588055788758) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7941499999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0796429418879614) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3798125) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07572344702548002) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6249750000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13772658944496724) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8243375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04642849444514059) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3520625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03889823328050272) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6363624999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10581089168215327) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98828125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.026208725517679103) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9966750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.01120071940534392) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9786374999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.032547605155072164) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9869625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.027445106555870208) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9994875000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003073578374100516) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9667125000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03659620343144428) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9929625) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.018372295927089957) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9515499999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.037149139284508935) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9968) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011264995674402745) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9349999999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03219017290513628) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9991749999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004546032142798204) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016472239047288895 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016472239047288895 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.875 - response_length/max:2394.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.875 - response_length_non_aborted/max:2394.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015249475836753845 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2428864315152168) - timing_s/agent_loop/generate_sequences/max:np.float64(17.01486107893288) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.637755410964019) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.01486107893288) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:2394 - timing_s/gen:23.984429011121392 - timing_s/reward:0.0665527768433094 - timing_s/old_log_prob:3.143460402265191 - timing_s/adv:0.6190033722668886 - timing_s/update_actor:22.729416826739907 - timing_s/step:50.62902601622045 - timing_s/testing:111.75804004445672 - timing_s/save_checkpoint:12.780955523252487 - timing_s/stop_profile:0.0001423358917236328 - timing_per_token_ms/gen:0.1788387989972664 - timing_per_token_ms/update_actor:0.11196757057507344 - timing_per_token_ms/adv:0.003049277695896003 - perf/total_num_tokens:203000 - perf/time_per_step:50.62902601622045 - perf/throughput:501.1947097672864 (TaskRunner pid=2320396) Training Progress: 10%|█ | 10/100 [11:29<2:16:44, 91.16s/it] (TaskRunner pid=2320396) step:11 - global_seqlen/min:20882 - global_seqlen/max:36506 - global_seqlen/minmax_diff:15624 - global_seqlen/balanced_min:26779 - global_seqlen/balanced_max:26829 - global_seqlen/mean:26804.75 - actor/entropy:0.2590366303920746 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016822338104248 - training/rollout_probs_diff_mean:0.002952962415292859 - training/rollout_probs_diff_std:0.008740789256989956 - training/rollout_actor_probs_pearson_corr:0.9993800520896912 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000083446502686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.9364482442324515e-06 - rollout_corr/rollout_is_max:1.9402036666870117 - rollout_corr/rollout_is_min:0.47953054308891296 - rollout_corr/rollout_is_std:0.027369895949959755 - rollout_corr/rollout_is_eff_sample_size:0.999251449571954 - rollout_corr/rollout_is_seq_mean:0.9999634027481079 - rollout_corr/rollout_is_seq_std:0.001723091583698988 - rollout_corr/rollout_is_seq_max:1.0086556673049927 - rollout_corr/rollout_is_seq_min:0.9935302734375 - rollout_corr/rollout_is_seq_max_deviation:0.008655667304992676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2878905679099262) - rollout_corr/training_log_ppl:np.float64(0.23974193456524517) - rollout_corr/kl:np.float64(0.0006918362495197672) - rollout_corr/k3_kl:np.float64(0.0005159022044267658) - rollout_corr/rollout_ppl:np.float64(1.287003616336733) - rollout_corr/rollout_log_ppl:np.float64(0.2390500977926422) - rollout_corr/log_ppl_diff:np.float64(0.0006918367726029828) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013698859984287992) - rollout_corr/log_ppl_diff_max:np.float64(0.014145970344543457) - rollout_corr/log_ppl_diff_min:np.float64(-0.007170185446739197) - rollout_corr/ppl_ratio:np.float64(1.0006940583698452) - rollout_corr/chi2_token:np.float64(0.0006612862925976515) - rollout_corr/chi2_seq:np.float64(0.25394133082590997) - actor/pg_loss:np.float64(0.0022024653153494) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030940706426463294) - actor/ppo_kl:np.float64(0.00023035600926712974) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(6060.7103929820005) - self_distillation/token_reweight_w_std:np.float64(178119.12218089768) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995679485145956) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06338222158956341) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.18062633275985718) - perf/mfu/actor:np.float64(0.06158845832096557) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.91942977905273) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010679529048502445 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010679529048502445 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:563.1484375 - response_length/max:2086.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:563.1484375 - response_length_non_aborted/max:2086.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.1552734375e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3518368285149336) - timing_s/agent_loop/generate_sequences/max:np.float64(16.211577968671918) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.0544039794185665) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.211577968671918) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:2063 - timing_s/gen:22.56175279431045 - timing_s/reward:0.06287666037678719 - timing_s/old_log_prob:3.2755117435008287 - timing_s/adv:0.7134620249271393 - timing_s/update_actor:22.722809998318553 - timing_s/step:49.44606627896428 - timing_s/stop_profile:0.00011546537280082703 - timing_per_token_ms/gen:0.1564984309359381 - timing_per_token_ms/update_actor:0.10596447457222392 - timing_per_token_ms/adv:0.0033271249728459473 - perf/total_num_tokens:214438 - perf/time_per_step:49.44606627896428 - perf/throughput:542.1007577988763 (TaskRunner pid=2320396) Training Progress: 11%|█ | 11/100 [12:18<1:56:18, 78.41s/it] (TaskRunner pid=2320396) step:12 - global_seqlen/min:18709 - global_seqlen/max:30849 - global_seqlen/minmax_diff:12140 - global_seqlen/balanced_min:25947 - global_seqlen/balanced_max:25976 - global_seqlen/mean:25967.5 - actor/entropy:0.2872362732887268 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19511616230010986 - training/rollout_probs_diff_mean:0.0031433426775038242 - training/rollout_probs_diff_std:0.008930543437600136 - training/rollout_actor_probs_pearson_corr:0.9993438720703125 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.5703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9999904632568359 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.423082358087413e-05 - rollout_corr/rollout_is_max:1.6279520988464355 - rollout_corr/rollout_is_min:0.480317085981369 - rollout_corr/rollout_is_std:0.028058120980858803 - rollout_corr/rollout_is_eff_sample_size:0.9992131230960879 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0013679719995707273 - rollout_corr/rollout_is_seq_max:1.0060700178146362 - rollout_corr/rollout_is_seq_min:0.9948147535324097 - rollout_corr/rollout_is_seq_max_deviation:0.0060700178146362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2835775869898498) - rollout_corr/training_log_ppl:np.float64(0.23835550290823448) - rollout_corr/kl:np.float64(0.00044148623862128034) - rollout_corr/k3_kl:np.float64(0.0004740920864492182) - rollout_corr/rollout_ppl:np.float64(1.2829798916354775) - rollout_corr/rollout_log_ppl:np.float64(0.23791401369089726) - rollout_corr/log_ppl_diff:np.float64(0.0004414892173372209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011379214993212372) - rollout_corr/log_ppl_diff_max:np.float64(0.006191082298755646) - rollout_corr/log_ppl_diff_min:np.float64(-0.00492994487285614) - rollout_corr/ppl_ratio:np.float64(1.0004427630919963) - rollout_corr/chi2_token:np.float64(0.0010108782444149256) - rollout_corr/chi2_seq:np.float64(0.6784807899966836) - actor/pg_loss:np.float64(0.0032307334477081895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003089785006977763) - actor/ppo_kl:np.float64(6.130353927158438e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5703125) - self_distillation/token_reweight_w_mean:np.float64(11269.828745884588) - self_distillation/token_reweight_w_std:np.float64(256559.5942551297) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003378372639418) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0678707295737695) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.19498711451888084) - perf/mfu/actor:np.float64(0.05924376023749836) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.71937561035156) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006124590989202261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006124590989202261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:548.984375 - response_length/max:2057.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:548.984375 - response_length_non_aborted/max:2057.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.933005690574646e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3749659731984138) - timing_s/agent_loop/generate_sequences/max:np.float64(15.617194678634405) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.826853218160977) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.617194678634405) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:2057 - timing_s/gen:22.177545238286257 - timing_s/reward:0.06833486817777157 - timing_s/old_log_prob:3.190919019281864 - timing_s/adv:0.6755244601517916 - timing_s/update_actor:22.886284664273262 - timing_s/step:49.08796023763716 - timing_s/stop_profile:0.00012080557644367218 - timing_per_token_ms/gen:0.15780237112769502 - timing_per_token_ms/update_actor:0.11016792463788035 - timing_per_token_ms/adv:0.003251778473822045 - perf/total_num_tokens:207740 - perf/time_per_step:49.08796023763716 - perf/throughput:528.9993691791244 (TaskRunner pid=2320396) Training Progress: 12%|█▏ | 12/100 [13:08<1:41:56, 69.51s/it] (TaskRunner pid=2320396) step:13 - global_seqlen/min:22787 - global_seqlen/max:30986 - global_seqlen/minmax_diff:8199 - global_seqlen/balanced_min:26391 - global_seqlen/balanced_max:26417 - global_seqlen/mean:26409.25 - actor/entropy:0.30309635400772095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41023290157318115 - training/rollout_probs_diff_mean:0.0032512678299099207 - training/rollout_probs_diff_std:0.00912945345044136 - training/rollout_actor_probs_pearson_corr:0.9993520975112915 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8828125 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8828125 - rollout_corr/rollout_is_mean:0.9999639391899109 - rollout_corr/rollout_is_ratio_fraction_high:7.1484737418359146e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.1484737418359146e-06 - rollout_corr/rollout_is_max:4.591783046722412 - rollout_corr/rollout_is_min:0.43883800506591797 - rollout_corr/rollout_is_std:0.028721565380692482 - rollout_corr/rollout_is_eff_sample_size:0.9991755136364232 - rollout_corr/rollout_is_seq_mean:0.9999345541000366 - rollout_corr/rollout_is_seq_std:0.001229618676006794 - rollout_corr/rollout_is_seq_max:1.0057193040847778 - rollout_corr/rollout_is_seq_min:0.9962084889411926 - rollout_corr/rollout_is_seq_max_deviation:0.005719304084777832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3604151918552816) - rollout_corr/training_log_ppl:np.float64(0.28699639250407927) - rollout_corr/kl:np.float64(0.000334256598858218) - rollout_corr/k3_kl:np.float64(0.0004752778168608529) - rollout_corr/rollout_ppl:np.float64(1.3599663516506553) - rollout_corr/rollout_log_ppl:np.float64(0.2866621360881254) - rollout_corr/log_ppl_diff:np.float64(0.0003342564159538597) - rollout_corr/log_ppl_abs_diff:np.float64(0.001109582808567211) - rollout_corr/log_ppl_diff_max:np.float64(0.004641130566596985) - rollout_corr/log_ppl_diff_min:np.float64(-0.003542080521583557) - rollout_corr/ppl_ratio:np.float64(1.0003352970816195) - rollout_corr/chi2_token:np.float64(0.0012653181329369545) - rollout_corr/chi2_seq:np.float64(0.7056709041353315) - actor/pg_loss:np.float64(0.003511187736876309) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003105056030108244) - actor/ppo_kl:np.float64(-0.0001611898588009808) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(4473.108445552643) - self_distillation/token_reweight_w_std:np.float64(105375.92507209204) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019248176831752) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10184741955890786) - self_distillation/empty_target_batch:np.float64(0.1171875) - actor/grad_norm:np.float64(0.26514898985624313) - perf/mfu/actor:np.float64(0.061133796091133896) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.83147811889648) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009976589120924473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009976589120924473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:546.4453125 - response_length/max:1478.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:546.4453125 - response_length_non_aborted/max:1478.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017240829765796661 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4114775881171227) - timing_s/agent_loop/generate_sequences/max:np.float64(12.42954215966165) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.215870589490805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.42954215966165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:363 - timing_s/agent_loop/slowest/response_length:1478 - timing_s/gen:19.216635648161173 - timing_s/reward:0.06841671280562878 - timing_s/old_log_prob:3.1306111067533493 - timing_s/adv:0.6264998782426119 - timing_s/update_actor:22.610267801210284 - timing_s/step:45.73972590826452 - timing_s/stop_profile:0.00011229142546653748 - timing_per_token_ms/gen:0.13736961647123577 - timing_per_token_ms/update_actor:0.1070186951598885 - timing_per_token_ms/adv:0.0029653430059667155 - perf/total_num_tokens:211274 - perf/time_per_step:45.73972590826452 - perf/throughput:577.3810287575034 (TaskRunner pid=2320396) Training Progress: 13%|█▎ | 13/100 [13:53<1:30:22, 62.33s/it] (TaskRunner pid=2320396) step:14 - global_seqlen/min:17015 - global_seqlen/max:32433 - global_seqlen/minmax_diff:15418 - global_seqlen/balanced_min:25084 - global_seqlen/balanced_max:25111 - global_seqlen/mean:25101.875 - actor/entropy:0.2703498601913452 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4553069472312927 - training/rollout_probs_diff_mean:0.003260517492890358 - training/rollout_probs_diff_std:0.00941763911396265 - training/rollout_actor_probs_pearson_corr:0.9992024898529053 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:7.63831030781148e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.527662061562296e-05 - rollout_corr/rollout_is_max:2.148858070373535 - rollout_corr/rollout_is_min:0.38240405917167664 - rollout_corr/rollout_is_std:0.029475368559360504 - rollout_corr/rollout_is_eff_sample_size:0.9991319568512997 - rollout_corr/rollout_is_seq_mean:1.0000320672988892 - rollout_corr/rollout_is_seq_std:0.0016445777146145701 - rollout_corr/rollout_is_seq_max:1.0072376728057861 - rollout_corr/rollout_is_seq_min:0.9936115741729736 - rollout_corr/rollout_is_seq_max_deviation:0.007237672805786133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3087067985907197) - rollout_corr/training_log_ppl:np.float64(0.2531808679632377) - rollout_corr/kl:np.float64(0.0006406861673453612) - rollout_corr/k3_kl:np.float64(0.0005106621201207417) - rollout_corr/rollout_ppl:np.float64(1.3078300086781383) - rollout_corr/rollout_log_ppl:np.float64(0.2525401824968867) - rollout_corr/log_ppl_diff:np.float64(0.0006406854663509876) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012924426409881562) - rollout_corr/log_ppl_diff_max:np.float64(0.007752135396003723) - rollout_corr/log_ppl_diff_min:np.float64(-0.005377739667892456) - rollout_corr/ppl_ratio:np.float64(1.0006423369050026) - rollout_corr/chi2_token:np.float64(0.0007580590900033712) - rollout_corr/chi2_seq:np.float64(0.2396695853676647) - actor/pg_loss:np.float64(0.004213686450384557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043405349424574524) - actor/ppo_kl:np.float64(0.0002313976682248864) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(1120.801469426835) - self_distillation/token_reweight_w_std:np.float64(23381.567357088614) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000115639762953) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08497079348308034) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2399461343884468) - perf/mfu/actor:np.float64(0.05799008474978051) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.85115432739258) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.018892407417297363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.018892407417297363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:511.40234375 - response_length/max:1582.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:511.40234375 - response_length_non_aborted/max:1582.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.643604815006256e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0278006736189127) - timing_s/agent_loop/generate_sequences/max:np.float64(12.826374592259526) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.649861370380677) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.826374592259526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:249 - timing_s/agent_loop/slowest/response_length:1582 - timing_s/gen:20.82091149315238 - timing_s/reward:0.06341278553009033 - timing_s/old_log_prob:3.1353542301803827 - timing_s/adv:0.6522118151187897 - timing_s/update_actor:22.489721328020096 - timing_s/step:47.25435267388821 - timing_s/stop_profile:0.00012648850679397583 - timing_per_token_ms/gen:0.15903659127515776 - timing_per_token_ms/update_actor:0.11199223826915368 - timing_per_token_ms/adv:0.0032478241920115016 - perf/total_num_tokens:200815 - perf/time_per_step:47.25435267388821 - perf/throughput:531.207678861524 (TaskRunner pid=2320396) Training Progress: 14%|█▍ | 14/100 [14:41<1:22:50, 57.80s/it] (TaskRunner pid=2320396) step:15 - global_seqlen/min:15276 - global_seqlen/max:29060 - global_seqlen/minmax_diff:13784 - global_seqlen/balanced_min:23721 - global_seqlen/balanced_max:23925 - global_seqlen/mean:23760.875 - actor/entropy:0.24135680496692657 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35413694381713867 - training/rollout_probs_diff_mean:0.002892824588343501 - training/rollout_probs_diff_std:0.009023798629641533 - training/rollout_actor_probs_pearson_corr:0.999297022819519 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999739527702332 - rollout_corr/rollout_is_ratio_fraction_high:8.203243851312436e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6406487702624872e-05 - rollout_corr/rollout_is_max:2.057389497756958 - rollout_corr/rollout_is_min:0.30289867520332336 - rollout_corr/rollout_is_std:0.02816413715481758 - rollout_corr/rollout_is_eff_sample_size:0.9992072910640203 - rollout_corr/rollout_is_seq_mean:1.0000672340393066 - rollout_corr/rollout_is_seq_std:0.0018630841514095664 - rollout_corr/rollout_is_seq_max:1.0082615613937378 - rollout_corr/rollout_is_seq_min:0.9932234883308411 - rollout_corr/rollout_is_seq_max_deviation:0.008261561393737793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2723506367765367) - rollout_corr/training_log_ppl:np.float64(0.2237588390125893) - rollout_corr/kl:np.float64(0.00038307747586419083) - rollout_corr/k3_kl:np.float64(0.00043506306940344075) - rollout_corr/rollout_ppl:np.float64(1.271862001158297) - rollout_corr/rollout_log_ppl:np.float64(0.22337575981509872) - rollout_corr/log_ppl_diff:np.float64(0.00038307919749058783) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012621376663446426) - rollout_corr/log_ppl_diff_max:np.float64(0.007576450705528259) - rollout_corr/log_ppl_diff_min:np.float64(-0.006189495325088501) - rollout_corr/ppl_ratio:np.float64(1.00038459058851) - rollout_corr/chi2_token:np.float64(0.0009704220574349165) - rollout_corr/chi2_seq:np.float64(0.41881759371608496) - actor/pg_loss:np.float64(0.0019638556987047195) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002059341913991375) - actor/ppo_kl:np.float64(2.1355168600578622e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(14600.623928362038) - self_distillation/token_reweight_w_std:np.float64(243156.56867134484) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005333223380148) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05767766255303286) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.17628217674791813) - perf/mfu/actor:np.float64(0.05489578998408176) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.80868911743164) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008619558066129684 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008619558066129684 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:476.18359375 - response_length/max:1702.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:476.18359375 - response_length_non_aborted/max:1702.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.830878555774689e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8086316455155611) - timing_s/agent_loop/generate_sequences/max:np.float64(13.160807492211461) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.187484379501257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.160807492211461) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1702 - timing_s/gen:20.94913785532117 - timing_s/reward:0.06456178799271584 - timing_s/old_log_prob:3.1010314859449863 - timing_s/adv:0.6535162795335054 - timing_s/update_actor:22.56844968162477 - timing_s/step:47.42496529221535 - timing_s/stop_profile:0.0001125577837228775 - timing_per_token_ms/gen:0.1718508802516851 - timing_per_token_ms/update_actor:0.11872694966844008 - timing_per_token_ms/adv:0.003437985130669143 - perf/total_num_tokens:190087 - perf/time_per_step:47.42496529221535 - perf/throughput:501.02039829853646 (TaskRunner pid=2320396) Training Progress: 15%|█▌ | 15/100 [15:28<1:17:28, 54.68s/it] (TaskRunner pid=2320396) step:16 - global_seqlen/min:18153 - global_seqlen/max:26318 - global_seqlen/minmax_diff:8165 - global_seqlen/balanced_min:22294 - global_seqlen/balanced_max:22350 - global_seqlen/mean:22311.75 - actor/entropy:0.2570214569568634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34247446060180664 - training/rollout_probs_diff_mean:0.003073042258620262 - training/rollout_probs_diff_std:0.00942500215023756 - training/rollout_actor_probs_pearson_corr:0.9992203712463379 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.76953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:1.0000674724578857 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6776629965752363e-05 - rollout_corr/rollout_is_max:1.8510113954544067 - rollout_corr/rollout_is_min:0.30873462557792664 - rollout_corr/rollout_is_std:0.029871081933379173 - rollout_corr/rollout_is_eff_sample_size:0.9991086329211257 - rollout_corr/rollout_is_seq_mean:1.0001213550567627 - rollout_corr/rollout_is_seq_std:0.0016576494090259075 - rollout_corr/rollout_is_seq_max:1.0090272426605225 - rollout_corr/rollout_is_seq_min:0.9953111410140991 - rollout_corr/rollout_is_seq_max_deviation:0.009027242660522461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2797890407964587) - rollout_corr/training_log_ppl:np.float64(0.23128712813195307) - rollout_corr/kl:np.float64(0.00038973505982442536) - rollout_corr/k3_kl:np.float64(0.000491941271206997) - rollout_corr/rollout_ppl:np.float64(1.2792712301015854) - rollout_corr/rollout_log_ppl:np.float64(0.23089739288843703) - rollout_corr/log_ppl_diff:np.float64(0.00038973524351604283) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012824619188904762) - rollout_corr/log_ppl_diff_max:np.float64(0.005027711391448975) - rollout_corr/log_ppl_diff_min:np.float64(-0.007322400808334351) - rollout_corr/ppl_ratio:np.float64(1.0003912099637091) - rollout_corr/chi2_token:np.float64(0.0011921904515475035) - rollout_corr/chi2_seq:np.float64(0.29298205487430096) - actor/pg_loss:np.float64(0.0026901537785306573) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002558936457717209) - actor/ppo_kl:np.float64(4.6419355181015476e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_w_mean:np.float64(25693.85642775311) - self_distillation/token_reweight_w_std:np.float64(431982.68447569985) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010720288846642) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060745136928744614) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.23966870829463005) - perf/mfu/actor:np.float64(0.05296940936700055) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.85827255249023) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012607329990714788 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012607329990714788 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.6484375 - response_length/max:1591.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.6484375 - response_length_non_aborted/max:1591.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013866089284420013 - timing_s/agent_loop/generate_sequences/min:np.float64(1.381975892931223) - timing_s/agent_loop/generate_sequences/max:np.float64(11.990967459976673) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.791547902925231) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.990967459976673) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:1591 - timing_s/gen:18.93207285180688 - timing_s/reward:0.05992904677987099 - timing_s/old_log_prob:3.1903146971017122 - timing_s/adv:0.584565894678235 - timing_s/update_actor:21.913405856117606 - timing_s/step:44.769447296857834 - timing_s/stop_profile:8.477084338665009e-05 - timing_per_token_ms/gen:0.16897903257650868 - timing_per_token_ms/update_actor:0.12276830513136355 - timing_per_token_ms/adv:0.0032749890454482227 - perf/total_num_tokens:178494 - perf/time_per_step:44.769447296857834 - perf/throughput:498.370012300017 (TaskRunner pid=2320396) Training Progress: 16%|█▌ | 16/100 [16:13<1:12:23, 51.71s/it] (TaskRunner pid=2320396) step:17 - global_seqlen/min:17426 - global_seqlen/max:28967 - global_seqlen/minmax_diff:11541 - global_seqlen/balanced_min:22692 - global_seqlen/balanced_max:22697 - global_seqlen/mean:22695.125 - actor/entropy:0.29208648204803467 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.279821515083313 - training/rollout_probs_diff_mean:0.003486455651000142 - training/rollout_probs_diff_std:0.00968213938176632 - training/rollout_actor_probs_pearson_corr:0.9992047548294067 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999521374702454 - rollout_corr/rollout_is_ratio_fraction_high:8.820131370157469e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.5134615898132324 - rollout_corr/rollout_is_min:0.5504810214042664 - rollout_corr/rollout_is_std:0.03140411525964737 - rollout_corr/rollout_is_eff_sample_size:0.999014634241615 - rollout_corr/rollout_is_seq_mean:1.0000044107437134 - rollout_corr/rollout_is_seq_std:0.0021442328579723835 - rollout_corr/rollout_is_seq_max:1.0077778100967407 - rollout_corr/rollout_is_seq_min:0.9927151203155518 - rollout_corr/rollout_is_seq_max_deviation:0.007777810096740723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3378607337363064) - rollout_corr/training_log_ppl:np.float64(0.27729781922244) - rollout_corr/kl:np.float64(0.0006814117384088547) - rollout_corr/k3_kl:np.float64(0.000675479077244745) - rollout_corr/rollout_ppl:np.float64(1.336908448021859) - rollout_corr/rollout_log_ppl:np.float64(0.27661640573933255) - rollout_corr/log_ppl_diff:np.float64(0.0006814134831074625) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016361892339773476) - rollout_corr/log_ppl_diff_max:np.float64(0.01305815577507019) - rollout_corr/log_ppl_diff_min:np.float64(-0.006799712777137756) - rollout_corr/ppl_ratio:np.float64(1.0006845023017377) - rollout_corr/chi2_token:np.float64(0.0012903332244604826) - rollout_corr/chi2_seq:np.float64(0.641126612201333) - actor/pg_loss:np.float64(0.0035220705904066563) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032299184999828867) - actor/ppo_kl:np.float64(0.00013300533843363382) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(1380.9805241154972) - self_distillation/token_reweight_w_std:np.float64(30178.533250448236) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999288574326783) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08566995974979363) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.26250072568655014) - perf/mfu/actor:np.float64(0.05411806700287069) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.87701416015625) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021399843972176313 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021399843972176313 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.87890625 - response_length/max:1282.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:442.87890625 - response_length_non_aborted/max:1282.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.532218635082245e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.001680351793766) - timing_s/agent_loop/generate_sequences/max:np.float64(10.440772091969848) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.841584001922456) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.440772091969848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:1282 - timing_s/gen:17.069299457594752 - timing_s/reward:0.06137548200786114 - timing_s/old_log_prob:3.0410669166594744 - timing_s/adv:0.6068573668599129 - timing_s/update_actor:21.700714791193604 - timing_s/step:42.56168849952519 - timing_s/stop_profile:0.00012058578431606293 - timing_per_token_ms/gen:0.15055345844037815 - timing_per_token_ms/update_actor:0.11952299663029838 - timing_per_token_ms/adv:0.003342443403924372 - perf/total_num_tokens:181561 - perf/time_per_step:42.56168849952519 - perf/throughput:533.228962480029 (TaskRunner pid=2320396) Training Progress: 17%|█▋ | 17/100 [16:56<1:07:44, 48.97s/it] (TaskRunner pid=2320396) step:18 - global_seqlen/min:18525 - global_seqlen/max:28543 - global_seqlen/minmax_diff:10018 - global_seqlen/balanced_min:24608 - global_seqlen/balanced_max:24629 - global_seqlen/mean:24623.75 - actor/entropy:0.291051983833313 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846030056476593 - training/rollout_probs_diff_mean:0.003182345302775502 - training/rollout_probs_diff_std:0.009170476347208023 - training/rollout_actor_probs_pearson_corr:0.9992757439613342 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.9999815821647644 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.633814811706543 - rollout_corr/rollout_is_min:0.6136082410812378 - rollout_corr/rollout_is_std:0.02904965728521347 - rollout_corr/rollout_is_eff_sample_size:0.9991567099682599 - rollout_corr/rollout_is_seq_mean:1.0001029968261719 - rollout_corr/rollout_is_seq_std:0.0016350590158253908 - rollout_corr/rollout_is_seq_max:1.0074173212051392 - rollout_corr/rollout_is_seq_min:0.9932276010513306 - rollout_corr/rollout_is_seq_max_deviation:0.00741732120513916 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3274645488709211) - rollout_corr/training_log_ppl:np.float64(0.2682839465123834) - rollout_corr/kl:np.float64(0.0003594352448370586) - rollout_corr/k3_kl:np.float64(0.00047769542220521544) - rollout_corr/rollout_ppl:np.float64(1.3270041062496603) - rollout_corr/rollout_log_ppl:np.float64(0.26792451113578863) - rollout_corr/log_ppl_diff:np.float64(0.0003594353765947744) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012344932329142466) - rollout_corr/log_ppl_diff_max:np.float64(0.008019745349884033) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066003501415252686) - rollout_corr/ppl_ratio:np.float64(1.0003609699197114) - rollout_corr/chi2_token:np.float64(0.0011884842533618212) - rollout_corr/chi2_seq:np.float64(0.5264976168982685) - actor/pg_loss:np.float64(0.003481487277895212) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036702533452626085) - actor/ppo_kl:np.float64(1.649830553063225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(23783.103630624246) - self_distillation/token_reweight_w_std:np.float64(391530.74363069737) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014330511912704) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09469849267043173) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.24629244580864906) - perf/mfu/actor:np.float64(0.057109274387849074) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.89093780517578) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014965659007430077 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014965659007430077 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:505.6171875 - response_length/max:1449.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:505.6171875 - response_length_non_aborted/max:1449.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002104882150888443 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9661016371101141) - timing_s/agent_loop/generate_sequences/max:np.float64(12.230161782354116) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.689887342115981) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.230161782354116) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:1449 - timing_s/gen:19.692222712561488 - timing_s/reward:0.06511449627578259 - timing_s/old_log_prob:3.095972517505288 - timing_s/adv:0.610324090346694 - timing_s/update_actor:22.506673527881503 - timing_s/step:46.059743424877524 - timing_s/stop_profile:0.00011930987238883972 - timing_per_token_ms/gen:0.15213633332221982 - timing_per_token_ms/update_actor:0.11425287338383422 - timing_per_token_ms/adv:0.00309824910069899 - perf/total_num_tokens:196990 - perf/time_per_step:46.059743424877524 - perf/throughput:534.6045845904639 (TaskRunner pid=2320396) Training Progress: 18%|█▊ | 18/100 [17:42<1:05:45, 48.11s/it] (TaskRunner pid=2320396) step:19 - global_seqlen/min:18204 - global_seqlen/max:34489 - global_seqlen/minmax_diff:16285 - global_seqlen/balanced_min:24308 - global_seqlen/balanced_max:24325 - global_seqlen/mean:24319.5 - actor/entropy:0.22665835916996002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28219449520111084 - training/rollout_probs_diff_mean:0.002949574962258339 - training/rollout_probs_diff_std:0.009281001053750515 - training/rollout_actor_probs_pearson_corr:0.9991229772567749 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.999921977519989 - rollout_corr/rollout_is_ratio_fraction_high:1.5755475033074617e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5755475033074617e-05 - rollout_corr/rollout_is_max:2.033278226852417 - rollout_corr/rollout_is_min:0.34410393238067627 - rollout_corr/rollout_is_std:0.028740236535668373 - rollout_corr/rollout_is_eff_sample_size:0.9991745615349487 - rollout_corr/rollout_is_seq_mean:0.999957263469696 - rollout_corr/rollout_is_seq_std:0.0018070649821311235 - rollout_corr/rollout_is_seq_max:1.008998155593872 - rollout_corr/rollout_is_seq_min:0.9937487840652466 - rollout_corr/rollout_is_seq_max_deviation:0.00899815559387207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.236382881179452) - rollout_corr/training_log_ppl:np.float64(0.20459451123315375) - rollout_corr/kl:np.float64(0.00044612464541060604) - rollout_corr/k3_kl:np.float64(0.00047015344770784395) - rollout_corr/rollout_ppl:np.float64(1.235796622466296) - rollout_corr/rollout_log_ppl:np.float64(0.20414838583383244) - rollout_corr/log_ppl_diff:np.float64(0.00044612539932131767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013511724537238479) - rollout_corr/log_ppl_diff_max:np.float64(0.00844089686870575) - rollout_corr/log_ppl_diff_min:np.float64(-0.005872577428817749) - rollout_corr/ppl_ratio:np.float64(1.000447863014415) - rollout_corr/chi2_token:np.float64(0.000981275923550129) - rollout_corr/chi2_seq:np.float64(0.4389119232073426) - actor/pg_loss:np.float64(0.0033063190057873726) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039961243192010443) - actor/ppo_kl:np.float64(-5.261532975175953e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(9821.60917656147) - self_distillation/token_reweight_w_std:np.float64(211790.16289496643) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992890893481672) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07317971339216456) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2147277519106865) - perf/mfu/actor:np.float64(0.05726284160695215) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.81607341766357) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015608752146363258 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015608752146363258 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:495.859375 - response_length/max:1619.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:495.859375 - response_length_non_aborted/max:1619.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011139735579490662 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4400303233414888) - timing_s/agent_loop/generate_sequences/max:np.float64(12.841846983879805) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.404720055048529) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.841846983879805) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1619 - timing_s/gen:19.706364968791604 - timing_s/reward:0.0602475106716156 - timing_s/old_log_prob:3.1028328090906143 - timing_s/adv:0.6127728633582592 - timing_s/update_actor:21.8852920550853 - timing_s/step:45.466720102354884 - timing_s/stop_profile:7.124058902263641e-05 - timing_per_token_ms/gen:0.1552415705750087 - timing_per_token_ms/update_actor:0.11248839437018289 - timing_per_token_ms/adv:0.0031495963288629453 - perf/total_num_tokens:194556 - perf/time_per_step:45.466720102354884 - perf/throughput:534.8857350002778 (TaskRunner pid=2320396) Training Progress: 19%|█▉ | 19/100 [18:27<1:03:53, 47.33s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2320396) (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The electric field due to a point charge is given by: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \times (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest answer is **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2320396) step:20 - global_seqlen/min:17811 - global_seqlen/max:27757 - global_seqlen/minmax_diff:9946 - global_seqlen/balanced_min:22789 - global_seqlen/balanced_max:22795 - global_seqlen/mean:22792.875 - actor/entropy:0.25308850407600403 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31527847051620483 - training/rollout_probs_diff_mean:0.0031136958859860897 - training/rollout_probs_diff_std:0.009490901604294777 - training/rollout_actor_probs_pearson_corr:0.9991628527641296 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000277757644653 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6279141902923584 - rollout_corr/rollout_is_min:0.5579100847244263 - rollout_corr/rollout_is_std:0.028953060507774353 - rollout_corr/rollout_is_eff_sample_size:0.9991623033905332 - rollout_corr/rollout_is_seq_mean:1.000030755996704 - rollout_corr/rollout_is_seq_std:0.001790462527424097 - rollout_corr/rollout_is_seq_max:1.0065312385559082 - rollout_corr/rollout_is_seq_min:0.9912646412849426 - rollout_corr/rollout_is_seq_max_deviation:0.008735358715057373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2623859508894384) - rollout_corr/training_log_ppl:np.float64(0.22125808593409602) - rollout_corr/kl:np.float64(0.0004030627003750453) - rollout_corr/k3_kl:np.float64(0.000487179084075251) - rollout_corr/rollout_ppl:np.float64(1.2618781537748873) - rollout_corr/rollout_log_ppl:np.float64(0.22085502157278825) - rollout_corr/log_ppl_diff:np.float64(0.00040306436130777) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013739361602347344) - rollout_corr/log_ppl_diff_max:np.float64(0.009082533419132233) - rollout_corr/log_ppl_diff_min:np.float64(-0.005283474922180176) - rollout_corr/ppl_ratio:np.float64(1.0004048922564834) - rollout_corr/chi2_token:np.float64(0.0011564446613192558) - rollout_corr/chi2_seq:np.float64(1.532239300897345) - actor/pg_loss:np.float64(0.003527254913933575) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003930750553990947) - actor/ppo_kl:np.float64(-8.912188441456692e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(4924.831674827263) - self_distillation/token_reweight_w_std:np.float64(105272.75272310019) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003256590571254) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07200192401069216) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2361677922308445) - perf/mfu/actor:np.float64(0.0536922197326441) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.8451690673828) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6171875) - val-aux/sciknoweval/reward/std@16:np.float64(0.202710008221792) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7003625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17668100954882657) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5318375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16688353844816986) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6167999999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20131107144087576) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7715250000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13401208491393962) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4669) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11627459214311622) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6370125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1686258674491296) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.82355) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08975117780023725) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4231249999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07250128837645714) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6521874999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13003496063039865) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8590249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05442224021498587) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.39719999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.041142257051020134) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6623249999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09169010950465917) - val-aux/sciknoweval/score/mean@16:np.float64(0.6171875) - val-aux/sciknoweval/score/std@16:np.float64(0.202710008221792) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7003625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17668100954882657) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5318375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16688353844816986) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6167999999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20131107144087576) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7715250000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13401208491393962) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4669) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11627459214311622) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6370125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1686258674491296) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.82355) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08975117780023725) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4231249999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07250128837645714) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6521874999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13003496063039865) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8590249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05442224021498587) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.39719999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.041142257051020134) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6623249999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09169010950465917) - val-core/sciknoweval/acc/mean@16:np.float64(0.6171875) - val-aux/sciknoweval/acc/std@16:np.float64(0.202710008221792) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7003625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17668100954882657) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5318375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16688353844816986) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6167999999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20131107144087576) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7715250000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13401208491393962) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4669) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11627459214311622) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6370125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1686258674491296) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.82355) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08975117780023725) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4231249999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07250128837645714) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6521874999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13003496063039865) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8590249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05442224021498587) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.39719999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.041142257051020134) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6623249999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09169010950465917) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.012252516113756723) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9973749999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006596626091455685) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.988175) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.014320320245906525) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.993) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0124317209813641) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995625) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0022972469936861388) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9827) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01450548118607801) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9957874999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.009331159818661627) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9774375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.01359035860325003) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9982875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.006036525988007402) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.971575) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.012021621146916916) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996500000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002062159062730128) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003338916227221489 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003338916227221489 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:452.46484375 - response_length/max:1161.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:452.46484375 - response_length_non_aborted/max:1161.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.982738316059113e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4439123030751944) - timing_s/agent_loop/generate_sequences/max:np.float64(9.813925538212061) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9232439590850845) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.813925538212061) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:1151 - timing_s/gen:16.630947606638074 - timing_s/reward:0.06120440363883972 - timing_s/old_log_prob:3.0552370827645063 - timing_s/adv:0.5773577280342579 - timing_s/update_actor:22.215389225631952 - timing_s/step:42.61977862007916 - timing_s/testing:104.57180244289339 - timing_s/save_checkpoint:12.73552224598825 - timing_s/stop_profile:0.00013989396393299103 - timing_per_token_ms/gen:0.14357941834774865 - timing_per_token_ms/update_actor:0.12183296987343606 - timing_per_token_ms/adv:0.0031663278987087953 - perf/total_num_tokens:182343 - perf/time_per_step:42.61977862007916 - perf/throughput:534.7957154630022 (TaskRunner pid=2320396) Training Progress: 20%|██ | 20/100 [21:07<1:48:12, 81.16s/it] (TaskRunner pid=2320396) step:21 - global_seqlen/min:21441 - global_seqlen/max:27978 - global_seqlen/minmax_diff:6537 - global_seqlen/balanced_min:23375 - global_seqlen/balanced_max:23403 - global_seqlen/mean:23390.75 - actor/entropy:0.21263273060321808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400324523448944 - training/rollout_probs_diff_mean:0.002692604437470436 - training/rollout_probs_diff_std:0.009109113365411758 - training/rollout_actor_probs_pearson_corr:0.9992377758026123 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999504089355469 - rollout_corr/rollout_is_ratio_fraction_high:1.6875073924893513e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.375014784978703e-05 - rollout_corr/rollout_is_max:2.1655259132385254 - rollout_corr/rollout_is_min:0.2829708158969879 - rollout_corr/rollout_is_std:0.0275435633957386 - rollout_corr/rollout_is_eff_sample_size:0.9992418081630519 - rollout_corr/rollout_is_seq_mean:0.9999712109565735 - rollout_corr/rollout_is_seq_std:0.0014922807458788157 - rollout_corr/rollout_is_seq_max:1.0057662725448608 - rollout_corr/rollout_is_seq_min:0.9951524138450623 - rollout_corr/rollout_is_seq_max_deviation:0.00576627254486084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2243655044585466) - rollout_corr/training_log_ppl:np.float64(0.191109212799347) - rollout_corr/kl:np.float64(0.00025866517576123016) - rollout_corr/k3_kl:np.float64(0.00041524350728394666) - rollout_corr/rollout_ppl:np.float64(1.2240168727003038) - rollout_corr/rollout_log_ppl:np.float64(0.19085054672905244) - rollout_corr/log_ppl_diff:np.float64(0.00025866607029456645) - rollout_corr/log_ppl_abs_diff:np.float64(0.001202312865643762) - rollout_corr/log_ppl_diff_max:np.float64(0.005112379789352417) - rollout_corr/log_ppl_diff_min:np.float64(-0.00403188169002533) - rollout_corr/ppl_ratio:np.float64(1.0002599137369543) - rollout_corr/chi2_token:np.float64(0.0011958186514675617) - rollout_corr/chi2_seq:np.float64(0.7327827827539295) - actor/pg_loss:np.float64(0.0018519839504733682) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023439306892214518) - actor/ppo_kl:np.float64(-0.00015845946509696063) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(232.10363201983273) - self_distillation/token_reweight_w_std:np.float64(5613.1256408341) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000109274405986) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04587709561747033) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.1534450314939022) - perf/mfu/actor:np.float64(0.05609792420550409) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(196.3541603088379) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004018377047032118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004018377047032118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:462.9609375 - response_length/max:1575.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:462.9609375 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001241639256477356 - timing_s/agent_loop/generate_sequences/min:np.float64(2.064604979008436) - timing_s/agent_loop/generate_sequences/max:np.float64(12.253852516412735) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.151852088274609) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.253852516412735) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:18.77224212884903 - timing_s/reward:0.05801803804934025 - timing_s/old_log_prob:3.214152639731765 - timing_s/adv:0.6185661982744932 - timing_s/update_actor:21.78351651877165 - timing_s/step:44.54678510688245 - timing_s/stop_profile:0.00012024678289890289 - timing_per_token_ms/gen:0.15839148592491462 - timing_per_token_ms/update_actor:0.11641095581999107 - timing_per_token_ms/adv:0.0033056133208345887 - perf/total_num_tokens:187126 - perf/time_per_step:44.54678510688245 - perf/throughput:525.0827852981504 (TaskRunner pid=2320396) Training Progress: 21%|██ | 21/100 [21:52<1:32:24, 70.19s/it] (TaskRunner pid=2320396) step:22 - global_seqlen/min:16968 - global_seqlen/max:30809 - global_seqlen/minmax_diff:13841 - global_seqlen/balanced_min:23449 - global_seqlen/balanced_max:23463 - global_seqlen/mean:23457.625 - actor/entropy:0.22844623029232025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.768605649471283 - training/rollout_probs_diff_mean:0.0029645978938788176 - training/rollout_probs_diff_std:0.009484783746302128 - training/rollout_actor_probs_pearson_corr:0.9990792870521545 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9999969601631165 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4725342882447876e-05 - rollout_corr/rollout_is_max:1.8854188919067383 - rollout_corr/rollout_is_min:0.12654437124729156 - rollout_corr/rollout_is_std:0.02832241728901863 - rollout_corr/rollout_is_eff_sample_size:0.9991984836344271 - rollout_corr/rollout_is_seq_mean:1.0000041723251343 - rollout_corr/rollout_is_seq_std:0.0015170971164479852 - rollout_corr/rollout_is_seq_max:1.0070810317993164 - rollout_corr/rollout_is_seq_min:0.993159294128418 - rollout_corr/rollout_is_seq_max_deviation:0.007081031799316406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2414707052521408) - rollout_corr/training_log_ppl:np.float64(0.20471240562619641) - rollout_corr/kl:np.float64(0.00047142372690922585) - rollout_corr/k3_kl:np.float64(0.0006191272363480493) - rollout_corr/rollout_ppl:np.float64(1.2409044564701617) - rollout_corr/rollout_log_ppl:np.float64(0.20424098065996077) - rollout_corr/log_ppl_diff:np.float64(0.0004714249662356451) - rollout_corr/log_ppl_abs_diff:np.float64(0.00149485188012477) - rollout_corr/log_ppl_diff_max:np.float64(0.007620379328727722) - rollout_corr/log_ppl_diff_min:np.float64(-0.0099639892578125) - rollout_corr/ppl_ratio:np.float64(1.000473550753668) - rollout_corr/chi2_token:np.float64(0.001530948095023632) - rollout_corr/chi2_seq:np.float64(1.3715695906430483) - actor/pg_loss:np.float64(0.0013342974707484245) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005119512775308976) - actor/ppo_kl:np.float64(4.771902877465095e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(48177.57957301638) - self_distillation/token_reweight_w_std:np.float64(715592.1982030283) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001572305103764) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04621635359944776) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.26206739246845245) - perf/mfu/actor:np.float64(0.05585179811843127) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(196.36561012268066) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010570083977654576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010570083977654576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:473.95703125 - response_length/max:1318.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:473.95703125 - response_length_non_aborted/max:1318.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001383572816848755 - timing_s/agent_loop/generate_sequences/min:np.float64(1.216639457270503) - timing_s/agent_loop/generate_sequences/max:np.float64(11.079553108662367) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.304368509227061) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.079553108662367) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1318 - timing_s/gen:17.77311465330422 - timing_s/reward:0.0648870412260294 - timing_s/old_log_prob:3.055829044431448 - timing_s/adv:0.5942137930542231 - timing_s/update_actor:21.682081462815404 - timing_s/step:43.26949073560536 - timing_s/stop_profile:0.00012202374637126923 - timing_per_token_ms/gen:0.14648211659898147 - timing_per_token_ms/update_actor:0.11553855869261809 - timing_per_token_ms/adv:0.0031664213291745383 - perf/total_num_tokens:187661 - perf/time_per_step:43.26949073560536 - perf/throughput:542.1285206090332 (TaskRunner pid=2320396) Training Progress: 22%|██▏ | 22/100 [22:35<1:20:45, 62.12s/it] (TaskRunner pid=2320396) step:23 - global_seqlen/min:17017 - global_seqlen/max:34235 - global_seqlen/minmax_diff:17218 - global_seqlen/balanced_min:25934 - global_seqlen/balanced_max:25946 - global_seqlen/mean:25942.125 - actor/entropy:0.26053303480148315 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3661079704761505 - training/rollout_probs_diff_mean:0.0031567697878926992 - training/rollout_probs_diff_std:0.009550051763653755 - training/rollout_actor_probs_pearson_corr:0.9991575479507446 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8515625 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8515625 - rollout_corr/rollout_is_mean:1.0000665187835693 - rollout_corr/rollout_is_ratio_fraction_high:1.4496010408038273e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.248005204019137e-06 - rollout_corr/rollout_is_max:2.1038994789123535 - rollout_corr/rollout_is_min:0.38240915536880493 - rollout_corr/rollout_is_std:0.02985311858355999 - rollout_corr/rollout_is_eff_sample_size:0.9991097038940658 - rollout_corr/rollout_is_seq_mean:1.0001295804977417 - rollout_corr/rollout_is_seq_std:0.0018321149982511997 - rollout_corr/rollout_is_seq_max:1.0110899209976196 - rollout_corr/rollout_is_seq_min:0.9960669875144958 - rollout_corr/rollout_is_seq_max_deviation:0.011089920997619629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.29484834196046) - rollout_corr/training_log_ppl:np.float64(0.2494393489760114) - rollout_corr/kl:np.float64(0.00033169940679478316) - rollout_corr/k3_kl:np.float64(0.0005022480274305963) - rollout_corr/rollout_ppl:np.float64(1.2943815663456917) - rollout_corr/rollout_log_ppl:np.float64(0.24910764716332778) - rollout_corr/log_ppl_diff:np.float64(0.00033170181268360466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013077733310637996) - rollout_corr/log_ppl_diff_max:np.float64(0.010803669691085815) - rollout_corr/log_ppl_diff_min:np.float64(-0.011415451765060425) - rollout_corr/ppl_ratio:np.float64(1.000333657488227) - rollout_corr/chi2_token:np.float64(0.0013864929787814617) - rollout_corr/chi2_seq:np.float64(0.9093728845473379) - actor/pg_loss:np.float64(0.0041787896770983934) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038857600884512067) - actor/ppo_kl:np.float64(-1.400662002115638e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(9336.005591196474) - self_distillation/token_reweight_w_std:np.float64(221650.8600916579) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002693643560633) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09581344586331397) - self_distillation/empty_target_batch:np.float64(0.1484375) - actor/grad_norm:np.float64(0.29776711761951447) - perf/mfu/actor:np.float64(0.060611924379589) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.15872192382812) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002980742137879133 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002980742137879133 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:538.94140625 - response_length/max:1397.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:538.94140625 - response_length_non_aborted/max:1397.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003536306321620941 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1290383003652096) - timing_s/agent_loop/generate_sequences/max:np.float64(11.685129918158054) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.777066974442278) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.685129918158054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1397 - timing_s/gen:18.946874292567372 - timing_s/reward:0.06363841146230698 - timing_s/old_log_prob:3.2305244877934456 - timing_s/adv:0.6419652253389359 - timing_s/update_actor:22.296060722321272 - timing_s/step:45.296368503943086 - timing_s/stop_profile:0.00012127123773097992 - timing_per_token_ms/gen:0.13732703935353138 - timing_per_token_ms/update_actor:0.10743173854455482 - timing_per_token_ms/adv:0.003093256746213619 - perf/total_num_tokens:207537 - perf/time_per_step:45.296368503943086 - perf/throughput:572.719753411175 (TaskRunner pid=2320396) Training Progress: 23%|██▎ | 23/100 [23:21<1:13:23, 57.19s/it] (TaskRunner pid=2320396) step:24 - global_seqlen/min:17547 - global_seqlen/max:27928 - global_seqlen/minmax_diff:10381 - global_seqlen/balanced_min:22587 - global_seqlen/balanced_max:22607 - global_seqlen/mean:22601.75 - actor/entropy:0.23046009242534637 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7216058373451233 - training/rollout_probs_diff_mean:0.0030117868445813656 - training/rollout_probs_diff_std:0.009528078138828278 - training/rollout_actor_probs_pearson_corr:0.9991067051887512 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999849200248718 - rollout_corr/rollout_is_ratio_fraction_high:8.861634341883473e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7723268683766946e-05 - rollout_corr/rollout_is_max:2.567455768585205 - rollout_corr/rollout_is_min:0.043630991131067276 - rollout_corr/rollout_is_std:0.028607197105884552 - rollout_corr/rollout_is_eff_sample_size:0.9991822974119504 - rollout_corr/rollout_is_seq_mean:0.9998600482940674 - rollout_corr/rollout_is_seq_std:0.0020710325334221125 - rollout_corr/rollout_is_seq_max:1.0104851722717285 - rollout_corr/rollout_is_seq_min:0.9908909201622009 - rollout_corr/rollout_is_seq_max_deviation:0.010485172271728516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2293109917081892) - rollout_corr/training_log_ppl:np.float64(0.19703222493990324) - rollout_corr/kl:np.float64(0.0006336333097620717) - rollout_corr/k3_kl:np.float64(0.0004862117235546748) - rollout_corr/rollout_ppl:np.float64(1.2285275124013424) - rollout_corr/rollout_log_ppl:np.float64(0.196398589774617) - rollout_corr/log_ppl_diff:np.float64(0.0006336351652862504) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014370774588314816) - rollout_corr/log_ppl_diff_max:np.float64(0.010363861918449402) - rollout_corr/log_ppl_diff_min:np.float64(-0.00926206260919571) - rollout_corr/ppl_ratio:np.float64(1.000636067474261) - rollout_corr/chi2_token:np.float64(0.0006539849564433098) - rollout_corr/chi2_seq:np.float64(0.4699943282175809) - actor/pg_loss:np.float64(0.0021280853543430567) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015898636911515496) - actor/ppo_kl:np.float64(2.8684361065955954e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(4608.7780102991965) - self_distillation/token_reweight_w_std:np.float64(110638.40136062028) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000740052200854) - self_distillation/token_reweight_w_clip_frac:np.float64(0.050635051098652184) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.1550346463918686) - perf/mfu/actor:np.float64(0.05328909417965846) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.19468307495117) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002256393665447831 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002256393665447831 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.8046875 - response_length/max:1444.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.8046875 - response_length_non_aborted/max:1444.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001823175698518753 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9695372581481934) - timing_s/agent_loop/generate_sequences/max:np.float64(11.480458768084645) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8320430894818855) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.480458768084645) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:1444 - timing_s/gen:18.5911016035825 - timing_s/reward:0.061533816158771515 - timing_s/old_log_prob:3.102441059425473 - timing_s/adv:0.6918395888060331 - timing_s/update_actor:21.863283349201083 - timing_s/step:44.40328153781593 - timing_s/stop_profile:0.00011542066931724548 - timing_per_token_ms/gen:0.16474754624517043 - timing_per_token_ms/update_actor:0.12091587680821775 - timing_per_token_ms/adv:0.0038262501178339793 - perf/total_num_tokens:180814 - perf/time_per_step:44.40328153781593 - perf/throughput:509.0108031937073 (TaskRunner pid=2320396) Training Progress: 24%|██▍ | 24/100 [24:05<1:07:36, 53.37s/it] (TaskRunner pid=2320396) step:25 - global_seqlen/min:18579 - global_seqlen/max:32122 - global_seqlen/minmax_diff:13543 - global_seqlen/balanced_min:24530 - global_seqlen/balanced_max:24557 - global_seqlen/mean:24549.25 - actor/entropy:0.2588360011577606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24279403686523438 - training/rollout_probs_diff_mean:0.00314454291947186 - training/rollout_probs_diff_std:0.00929185375571251 - training/rollout_actor_probs_pearson_corr:0.9992793798446655 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000556707382202 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.201383151463233e-05 - rollout_corr/rollout_is_max:1.6370543241500854 - rollout_corr/rollout_is_min:0.29653897881507874 - rollout_corr/rollout_is_std:0.028675872832536697 - rollout_corr/rollout_is_eff_sample_size:0.999178488965226 - rollout_corr/rollout_is_seq_mean:1.0000877380371094 - rollout_corr/rollout_is_seq_std:0.001914774882607162 - rollout_corr/rollout_is_seq_max:1.0101841688156128 - rollout_corr/rollout_is_seq_min:0.9926535487174988 - rollout_corr/rollout_is_seq_max_deviation:0.010184168815612793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2983261444605887) - rollout_corr/training_log_ppl:np.float64(0.2460111120599322) - rollout_corr/kl:np.float64(0.0005979655367260861) - rollout_corr/k3_kl:np.float64(0.0004908691531397835) - rollout_corr/rollout_ppl:np.float64(1.297467083670199) - rollout_corr/rollout_log_ppl:np.float64(0.24541314343514387) - rollout_corr/log_ppl_diff:np.float64(0.0005979686247883365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014427673449972644) - rollout_corr/log_ppl_diff_max:np.float64(0.009425044059753418) - rollout_corr/log_ppl_diff_min:np.float64(-0.006913349032402039) - rollout_corr/ppl_ratio:np.float64(1.0006001458968967) - rollout_corr/chi2_token:np.float64(0.0007500855717808008) - rollout_corr/chi2_seq:np.float64(0.46263502025976777) - actor/pg_loss:np.float64(0.0036563018802553415) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040037721146291005) - actor/ppo_kl:np.float64(0.00023973096506502145) - actor/pg_clipfrac_lower:np.float64(7.063743396429345e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(5786.897568458458) - self_distillation/token_reweight_w_std:np.float64(129991.75501095905) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001443214714527) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07592839169956278) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2528732307255268) - perf/mfu/actor:np.float64(0.058185086481743295) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.12001419067383) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021869447082281113 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021869447082281113 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:488.0703125 - response_length/max:1537.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:488.0703125 - response_length_non_aborted/max:1537.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014443323016166687 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8763598762452602) - timing_s/agent_loop/generate_sequences/max:np.float64(12.1195951346308) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.420748021701002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.1195951346308) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:1537 - timing_s/gen:19.121401023119688 - timing_s/reward:0.06362378038465977 - timing_s/old_log_prob:3.279709119349718 - timing_s/adv:0.657275116071105 - timing_s/update_actor:22.008095744997263 - timing_s/step:45.220784513279796 - timing_s/stop_profile:0.00011463277041912079 - timing_per_token_ms/gen:0.15303732030733028 - timing_per_token_ms/update_actor:0.11206093742679137 - timing_per_token_ms/adv:0.003346716885806618 - perf/total_num_tokens:196394 - perf/time_per_step:45.220784513279796 - perf/throughput:542.8753672504449 (TaskRunner pid=2320396) Training Progress: 25%|██▌ | 25/100 [24:51<1:03:40, 50.94s/it] (TaskRunner pid=2320396) step:26 - global_seqlen/min:19341 - global_seqlen/max:24825 - global_seqlen/minmax_diff:5484 - global_seqlen/balanced_min:21767 - global_seqlen/balanced_max:21774 - global_seqlen/mean:21770.625 - actor/entropy:0.2076856642961502 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105889558792114 - training/rollout_probs_diff_mean:0.002844884293153882 - training/rollout_probs_diff_std:0.009245309047400951 - training/rollout_actor_probs_pearson_corr:0.9991264343261719 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9999702572822571 - rollout_corr/rollout_is_ratio_fraction_high:9.282895916840062e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8565791833680123e-05 - rollout_corr/rollout_is_max:2.44268798828125 - rollout_corr/rollout_is_min:0.3941245973110199 - rollout_corr/rollout_is_std:0.027870552614331245 - rollout_corr/rollout_is_eff_sample_size:0.9992238351690815 - rollout_corr/rollout_is_seq_mean:1.0000226497650146 - rollout_corr/rollout_is_seq_std:0.0018026805482804775 - rollout_corr/rollout_is_seq_max:1.0095715522766113 - rollout_corr/rollout_is_seq_min:0.9939136505126953 - rollout_corr/rollout_is_seq_max_deviation:0.009571552276611328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2315654191188514) - rollout_corr/training_log_ppl:np.float64(0.19640630582580343) - rollout_corr/kl:np.float64(0.0003411613332708896) - rollout_corr/k3_kl:np.float64(0.00045860192925317733) - rollout_corr/rollout_ppl:np.float64(1.2310992125421762) - rollout_corr/rollout_log_ppl:np.float64(0.19606514342012815) - rollout_corr/log_ppl_diff:np.float64(0.0003411624056752771) - rollout_corr/log_ppl_abs_diff:np.float64(0.001351964398054406) - rollout_corr/log_ppl_diff_max:np.float64(0.007199376821517944) - rollout_corr/log_ppl_diff_min:np.float64(-0.008631736040115356) - rollout_corr/ppl_ratio:np.float64(1.0003430233336985) - rollout_corr/chi2_token:np.float64(0.0012167126405984163) - rollout_corr/chi2_seq:np.float64(0.3726298357360065) - actor/pg_loss:np.float64(0.001817713608033955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024271494748973055) - actor/ppo_kl:np.float64(-6.930642252278574e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(21209.821032542968) - self_distillation/token_reweight_w_std:np.float64(407491.57075386524) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005513478536159) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04697586127440445) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.18191703036427498) - perf/mfu/actor:np.float64(0.052533244830375574) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.11370468139648) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007448364049196243 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007448364049196243 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.80078125 - response_length/max:1230.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.80078125 - response_length_non_aborted/max:1230.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011184252798557281 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7711089085787535) - timing_s/agent_loop/generate_sequences/max:np.float64(9.832402851432562) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.4123942583973985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.832402851432562) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1202 - timing_s/gen:16.367364613339305 - timing_s/reward:0.06243230774998665 - timing_s/old_log_prob:3.00197671353817 - timing_s/adv:0.577815530821681 - timing_s/update_actor:21.668468771502376 - timing_s/step:41.76663071103394 - timing_s/stop_profile:0.00011580996215343475 - timing_per_token_ms/gen:0.15193654781470695 - timing_per_token_ms/update_actor:0.12441345144835286 - timing_per_token_ms/adv:0.003317632881587466 - perf/total_num_tokens:174165 - perf/time_per_step:41.76663071103394 - perf/throughput:521.2444630887743 (TaskRunner pid=2320396) Training Progress: 26%|██▌ | 26/100 [25:32<59:27, 48.20s/it] (TaskRunner pid=2320396) step:27 - global_seqlen/min:20266 - global_seqlen/max:29377 - global_seqlen/minmax_diff:9111 - global_seqlen/balanced_min:25441 - global_seqlen/balanced_max:25448 - global_seqlen/mean:25444.0 - actor/entropy:0.21156783401966095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2872914671897888 - training/rollout_probs_diff_mean:0.002737778937444091 - training/rollout_probs_diff_std:0.009022722952067852 - training/rollout_actor_probs_pearson_corr:0.9992183446884155 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.9921875 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:0.9998476505279541 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6285487413406372 - rollout_corr/rollout_is_min:0.5106052756309509 - rollout_corr/rollout_is_std:0.02745903842151165 - rollout_corr/rollout_is_eff_sample_size:0.9992465693293799 - rollout_corr/rollout_is_seq_mean:0.9997648000717163 - rollout_corr/rollout_is_seq_std:0.0015325150452554226 - rollout_corr/rollout_is_seq_max:1.0047353506088257 - rollout_corr/rollout_is_seq_min:0.9945488572120667 - rollout_corr/rollout_is_seq_max_deviation:0.00545114278793335 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2360313851386309) - rollout_corr/training_log_ppl:np.float64(0.20034451349056326) - rollout_corr/kl:np.float64(0.0006052363555104634) - rollout_corr/k3_kl:np.float64(0.00048079977509019045) - rollout_corr/rollout_ppl:np.float64(1.2352807368151844) - rollout_corr/rollout_log_ppl:np.float64(0.19973927503451705) - rollout_corr/log_ppl_diff:np.float64(0.0006052384560462087) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012537067814264446) - rollout_corr/log_ppl_diff_max:np.float64(0.005964487791061401) - rollout_corr/log_ppl_diff_min:np.float64(-0.004271447658538818) - rollout_corr/ppl_ratio:np.float64(1.0006067522335798) - rollout_corr/chi2_token:np.float64(0.0007339166477322578) - rollout_corr/chi2_seq:np.float64(0.17413755250163376) - actor/pg_loss:np.float64(0.0029487464344128966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036103266734244244) - actor/ppo_kl:np.float64(-5.398572019643666e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(677.8268313717563) - self_distillation/token_reweight_w_std:np.float64(18296.45410767995) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002783685922623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0762047362368321) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.38731055706739426) - perf/mfu/actor:np.float64(0.05949076540086539) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.24272918701172) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0031912531703710556 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0031912531703710556 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:518.84375 - response_length/max:1403.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:518.84375 - response_length_non_aborted/max:1403.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010640919208526611 - timing_s/agent_loop/generate_sequences/min:np.float64(1.03832764364779) - timing_s/agent_loop/generate_sequences/max:np.float64(11.360929055139422) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.670121224604372) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.360929055139422) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:1403 - timing_s/gen:17.88263091444969 - timing_s/reward:0.06664908863604069 - timing_s/old_log_prob:3.0836950354278088 - timing_s/adv:0.6352357696741819 - timing_s/update_actor:22.302965883165598 - timing_s/step:44.05930548161268 - timing_s/stop_profile:5.204230546951294e-05 - timing_per_token_ms/gen:0.13463403386774747 - timing_per_token_ms/update_actor:0.10956888600045982 - timing_per_token_ms/adv:0.003120754252840463 - perf/total_num_tokens:203552 - perf/time_per_step:44.05930548161268 - perf/throughput:577.494350441329 (TaskRunner pid=2320396) Training Progress: 27%|██▋ | 27/100 [26:16<57:08, 46.97s/it] (TaskRunner pid=2320396) step:28 - global_seqlen/min:17086 - global_seqlen/max:33653 - global_seqlen/minmax_diff:16567 - global_seqlen/balanced_min:24718 - global_seqlen/balanced_max:24765 - global_seqlen/mean:24755.125 - actor/entropy:0.24302850663661957 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4545377194881439 - training/rollout_probs_diff_mean:0.002987111918628216 - training/rollout_probs_diff_std:0.009371647611260414 - training/rollout_actor_probs_pearson_corr:0.9992448687553406 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000720024108887 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5652269212296233e-05 - rollout_corr/rollout_is_max:1.6478630304336548 - rollout_corr/rollout_is_min:0.32801195979118347 - rollout_corr/rollout_is_std:0.02821488305926323 - rollout_corr/rollout_is_eff_sample_size:0.9992046726227877 - rollout_corr/rollout_is_seq_mean:0.999945878982544 - rollout_corr/rollout_is_seq_std:0.0020114099606871605 - rollout_corr/rollout_is_seq_max:1.0073790550231934 - rollout_corr/rollout_is_seq_min:0.987631618976593 - rollout_corr/rollout_is_seq_max_deviation:0.012368381023406982 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2567434725351632) - rollout_corr/training_log_ppl:np.float64(0.21964200896036346) - rollout_corr/kl:np.float64(0.0004823495658996535) - rollout_corr/k3_kl:np.float64(0.0004897397312362273) - rollout_corr/rollout_ppl:np.float64(1.256141950841993) - rollout_corr/rollout_log_ppl:np.float64(0.21915965757216327) - rollout_corr/log_ppl_diff:np.float64(0.0004823513882001862) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014439801598200575) - rollout_corr/log_ppl_diff_max:np.float64(0.015101104974746704) - rollout_corr/log_ppl_diff_min:np.float64(-0.006693243980407715) - rollout_corr/ppl_ratio:np.float64(1.000484771328047) - rollout_corr/chi2_token:np.float64(0.0009968443773686886) - rollout_corr/chi2_seq:np.float64(1.4560303187463433) - actor/pg_loss:np.float64(0.0032396138412877917) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000316952940693227) - actor/ppo_kl:np.float64(-2.395193692983355e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(23524.618457379518) - self_distillation/token_reweight_w_std:np.float64(496158.66897797544) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004901748616248) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08251259801909328) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.25753332301974297) - perf/mfu/actor:np.float64(0.05782911225939355) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.28218793869019) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01441084872931242 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01441084872931242 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:499.12890625 - response_length/max:1897.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.12890625 - response_length_non_aborted/max:1897.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001032557338476181 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9153902158141136) - timing_s/agent_loop/generate_sequences/max:np.float64(14.23232613503933) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.135112953190401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.23232613503933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1897 - timing_s/gen:21.24835624359548 - timing_s/reward:0.06606246717274189 - timing_s/old_log_prob:3.0791484899818897 - timing_s/adv:0.6036070547997952 - timing_s/update_actor:22.402966856956482 - timing_s/step:47.48915730416775 - timing_s/stop_profile:0.00010910257697105408 - timing_per_token_ms/gen:0.16629249586072203 - timing_per_token_ms/update_actor:0.11312287282409442 - timing_per_token_ms/adv:0.0030478893501840283 - perf/total_num_tokens:198041 - perf/time_per_step:47.48915730416775 - perf/throughput:521.279517373694 (TaskRunner pid=2320396) Training Progress: 28%|██▊ | 28/100 [27:04<56:33, 47.14s/it] (TaskRunner pid=2320396) step:29 - global_seqlen/min:20180 - global_seqlen/max:30311 - global_seqlen/minmax_diff:10131 - global_seqlen/balanced_min:24016 - global_seqlen/balanced_max:24053 - global_seqlen/mean:24044.5 - actor/entropy:0.22302956879138947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6587663888931274 - training/rollout_probs_diff_mean:0.002952124923467636 - training/rollout_probs_diff_std:0.009483864530920982 - training/rollout_actor_probs_pearson_corr:0.9991285800933838 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9999752640724182 - rollout_corr/rollout_is_ratio_fraction_high:2.402075369900558e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.006917596503627e-06 - rollout_corr/rollout_is_max:3.0540003776550293 - rollout_corr/rollout_is_min:0.08800696581602097 - rollout_corr/rollout_is_std:0.02860928140580654 - rollout_corr/rollout_is_eff_sample_size:0.9991820593831796 - rollout_corr/rollout_is_seq_mean:0.9998377561569214 - rollout_corr/rollout_is_seq_std:0.0019047566456720233 - rollout_corr/rollout_is_seq_max:1.0071954727172852 - rollout_corr/rollout_is_seq_min:0.9904940724372864 - rollout_corr/rollout_is_seq_max_deviation:0.009505927562713623 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.239888684824109) - rollout_corr/training_log_ppl:np.float64(0.2059190910149482) - rollout_corr/kl:np.float64(0.0005806351006372168) - rollout_corr/k3_kl:np.float64(0.0005054707310918616) - rollout_corr/rollout_ppl:np.float64(1.2391317044384778) - rollout_corr/rollout_log_ppl:np.float64(0.20533845498721348) - rollout_corr/log_ppl_diff:np.float64(0.0005806360277347267) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014441340463235974) - rollout_corr/log_ppl_diff_max:np.float64(0.009319275617599487) - rollout_corr/log_ppl_diff_min:np.float64(-0.008508995175361633) - rollout_corr/ppl_ratio:np.float64(1.0005828321445733) - rollout_corr/chi2_token:np.float64(0.0008565832395106554) - rollout_corr/chi2_seq:np.float64(0.48222310887649655) - actor/pg_loss:np.float64(0.002814142149873078) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031159500281319197) - actor/ppo_kl:np.float64(-3.463015074700593e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(1034.8058356901165) - self_distillation/token_reweight_w_std:np.float64(28329.355002246273) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994706646539271) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05690540156501811) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.188645763322711) - perf/mfu/actor:np.float64(0.05621638661845592) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.2503433227539) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013596748001873493 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013596748001873493 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:487.859375 - response_length/max:1808.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:487.859375 - response_length_non_aborted/max:1808.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.099580347537994e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1761888433247805) - timing_s/agent_loop/generate_sequences/max:np.float64(13.156388711184263) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.122582192576374) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.156388711184263) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:1808 - timing_s/gen:20.01565883681178 - timing_s/reward:0.06360073015093803 - timing_s/old_log_prob:3.1809177827090025 - timing_s/adv:0.6148683913052082 - timing_s/update_actor:22.266462163999677 - timing_s/step:46.22870226763189 - timing_s/stop_profile:2.8520822525024414e-05 - timing_per_token_ms/gen:0.16026373856461407 - timing_per_token_ms/update_actor:0.11575652521366464 - timing_per_token_ms/adv:0.003196512670804177 - perf/total_num_tokens:192356 - perf/time_per_step:46.22870226763189 - perf/throughput:520.1205921983088 (TaskRunner pid=2320396) Training Progress: 29%|██▉ | 29/100 [27:50<55:27, 46.87s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) We solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \times (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2320396) step:30 - global_seqlen/min:17742 - global_seqlen/max:26480 - global_seqlen/minmax_diff:8738 - global_seqlen/balanced_min:22366 - global_seqlen/balanced_max:22378 - global_seqlen/mean:22373.5 - actor/entropy:0.24863006174564362 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24456164240837097 - training/rollout_probs_diff_mean:0.0031601106747984886 - training/rollout_probs_diff_std:0.009502153843641281 - training/rollout_actor_probs_pearson_corr:0.9991329312324524 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0000206232070923 - rollout_corr/rollout_is_ratio_fraction_high:1.776640783646144e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.664961175469216e-05 - rollout_corr/rollout_is_max:3.497854709625244 - rollout_corr/rollout_is_min:0.16539886593818665 - rollout_corr/rollout_is_std:0.029274486005306244 - rollout_corr/rollout_is_eff_sample_size:0.9991437382299739 - rollout_corr/rollout_is_seq_mean:0.9998601078987122 - rollout_corr/rollout_is_seq_std:0.0016287954058498144 - rollout_corr/rollout_is_seq_max:1.0044320821762085 - rollout_corr/rollout_is_seq_min:0.992756724357605 - rollout_corr/rollout_is_seq_max_deviation:0.0072432756423950195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2682397873140872) - rollout_corr/training_log_ppl:np.float64(0.22362850421632174) - rollout_corr/kl:np.float64(0.00048620960521006396) - rollout_corr/k3_kl:np.float64(0.0004756366115827859) - rollout_corr/rollout_ppl:np.float64(1.2676585195586085) - rollout_corr/rollout_log_ppl:np.float64(0.22314229208859615) - rollout_corr/log_ppl_diff:np.float64(0.00048621212772559375) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014107342140050605) - rollout_corr/log_ppl_diff_max:np.float64(0.007874712347984314) - rollout_corr/log_ppl_diff_min:np.float64(-0.010280773043632507) - rollout_corr/ppl_ratio:np.float64(1.0004882055800408) - rollout_corr/chi2_token:np.float64(0.0009464134927839041) - rollout_corr/chi2_seq:np.float64(0.9949460891075432) - actor/pg_loss:np.float64(0.002644557738676667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018046644299829495) - actor/ppo_kl:np.float64(-0.00010434456421570815) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(15509.348625806859) - self_distillation/token_reweight_w_std:np.float64(335266.9792876573) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003502645995468) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06214348939829506) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.1956712994724512) - perf/mfu/actor:np.float64(0.05303688677384744) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.29561233520508) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6140625) - val-aux/sciknoweval/reward/std@16:np.float64(0.20064917766957618) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6982125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17558868106714795) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.528725) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1636992413789501) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6131) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20074825670766394) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.77105) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13181042805409973) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46654999999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11489070749110727) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6290749999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16748543115238934) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.82365) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08642785589755841) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42318750000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07302527970826564) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6379374999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12699065808758647) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8556874999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04432075210710901) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3948625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04169575957732149) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6434875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09359792416990569) - val-aux/sciknoweval/score/mean@16:np.float64(0.6140625) - val-aux/sciknoweval/score/std@16:np.float64(0.20064917766957618) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6982125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17558868106714795) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.528725) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1636992413789501) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6131) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20074825670766394) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.77105) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13181042805409973) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46654999999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11489070749110727) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6290749999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16748543115238934) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.82365) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08642785589755841) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42318750000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07302527970826564) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6379374999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12699065808758647) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8556874999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04432075210710901) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3948625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04169575957732149) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6434875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09359792416990569) - val-core/sciknoweval/acc/mean@16:np.float64(0.6140625) - val-aux/sciknoweval/acc/std@16:np.float64(0.20064917766957618) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6982125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17558868106714795) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.528725) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1636992413789501) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6131) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20074825670766394) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.77105) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13181042805409973) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46654999999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11489070749110727) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6290749999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16748543115238934) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.82365) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08642785589755841) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42318750000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07302527970826564) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6379374999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12699065808758647) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8556874999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04432075210710901) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3948625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04169575957732149) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6434875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09359792416990569) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.028446172895248756) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.998475) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.00989590809890792) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9818999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.036282721326102566) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9904250000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.027632795745506478) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0013474443894951921) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9682624999999998) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04318228991168178) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9961500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01706630999890189) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.94895) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.045543356359162894) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008950291396294888) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9274875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03923536913417337) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999125000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0016718450301930732) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00767730874940753 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00767730874940753 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.734375 - response_length/max:1411.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.734375 - response_length_non_aborted/max:1411.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.119906902313232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9468586817383766) - timing_s/agent_loop/generate_sequences/max:np.float64(11.205822797492146) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.667787884332938) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.205822797492146) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1407 - timing_s/gen:18.225317746400833 - timing_s/reward:0.06284921430051327 - timing_s/old_log_prob:3.0297752507030964 - timing_s/adv:0.6036888118833303 - timing_s/update_actor:22.02555533312261 - timing_s/step:44.033918507397175 - timing_s/testing:111.50396230816841 - timing_s/save_checkpoint:13.47470897063613 - timing_s/stop_profile:5.4797157645225525e-05 - timing_per_token_ms/gen:0.16189920891874385 - timing_per_token_ms/update_actor:0.12305604472435365 - timing_per_token_ms/adv:0.003372789303659074 - perf/total_num_tokens:178988 - perf/time_per_step:44.033918507397175 - perf/throughput:508.0969570364608 (TaskRunner pid=2320396) Training Progress: 30%|███ | 30/100 [30:39<1:37:26, 83.53s/it] (TaskRunner pid=2320396) step:31 - global_seqlen/min:19933 - global_seqlen/max:31808 - global_seqlen/minmax_diff:11875 - global_seqlen/balanced_min:24800 - global_seqlen/balanced_max:24806 - global_seqlen/mean:24803.625 - actor/entropy:0.23857657611370087 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21513324975967407 - training/rollout_probs_diff_mean:0.0027538202702999115 - training/rollout_probs_diff_std:0.008704227395355701 - training/rollout_actor_probs_pearson_corr:0.9992588758468628 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9999880194664001 - rollout_corr/rollout_is_ratio_fraction_high:7.856385309423786e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.571277061884757e-05 - rollout_corr/rollout_is_max:2.55385684967041 - rollout_corr/rollout_is_min:0.49398577213287354 - rollout_corr/rollout_is_std:0.027300119400024414 - rollout_corr/rollout_is_eff_sample_size:0.9992551395431086 - rollout_corr/rollout_is_seq_mean:0.9999855756759644 - rollout_corr/rollout_is_seq_std:0.0014757115859538317 - rollout_corr/rollout_is_seq_max:1.0052125453948975 - rollout_corr/rollout_is_seq_min:0.995438277721405 - rollout_corr/rollout_is_seq_max_deviation:0.005212545394897461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2466215244494379) - rollout_corr/training_log_ppl:np.float64(0.20910465411725454) - rollout_corr/kl:np.float64(0.0004072134899462121) - rollout_corr/k3_kl:np.float64(0.0004494452254277803) - rollout_corr/rollout_ppl:np.float64(1.2461073002777994) - rollout_corr/rollout_log_ppl:np.float64(0.20869743956427556) - rollout_corr/log_ppl_diff:np.float64(0.000407214552978985) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012124801141908392) - rollout_corr/log_ppl_diff_max:np.float64(0.006325840950012207) - rollout_corr/log_ppl_diff_min:np.float64(-0.004628986120223999) - rollout_corr/ppl_ratio:np.float64(1.0004085870459676) - rollout_corr/chi2_token:np.float64(0.0010155059862881899) - rollout_corr/chi2_seq:np.float64(0.6279427350964397) - actor/pg_loss:np.float64(0.002768305013887584) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030953091891205986) - actor/ppo_kl:np.float64(2.230859727170298e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(27233.19396411348) - self_distillation/token_reweight_w_std:np.float64(526200.2407925605) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011748706456274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06611918151611462) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.22732961922883987) - perf/mfu/actor:np.float64(0.057831701660071295) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(199.57726669311523) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005538751720450819 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005538751720450819 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:497.20703125 - response_length/max:1315.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:497.20703125 - response_length_non_aborted/max:1315.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.376819849014282e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8423037379980087) - timing_s/agent_loop/generate_sequences/max:np.float64(11.606234759092331) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.555152973211079) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.606234759092331) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:1315 - timing_s/gen:18.41877550072968 - timing_s/reward:0.05963750369846821 - timing_s/old_log_prob:3.2094839587807655 - timing_s/adv:0.6232157163321972 - timing_s/update_actor:22.237760320305824 - timing_s/step:44.65447077155113 - timing_s/stop_profile:0.00010682083666324615 - timing_per_token_ms/gen:0.1447049966667689 - timing_per_token_ms/update_actor:0.11206910441672248 - timing_per_token_ms/adv:0.0031407491663627655 - perf/total_num_tokens:198429 - perf/time_per_step:44.65447077155113 - perf/throughput:555.4567005595801 (TaskRunner pid=2320396) Training Progress: 31%|███ | 31/100 [31:24<1:22:39, 71.88s/it] (TaskRunner pid=2320396) step:32 - global_seqlen/min:14922 - global_seqlen/max:26458 - global_seqlen/minmax_diff:11536 - global_seqlen/balanced_min:21089 - global_seqlen/balanced_max:21197 - global_seqlen/mean:21116.75 - actor/entropy:0.23724991083145142 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22528886795043945 - training/rollout_probs_diff_mean:0.003095978172495961 - training/rollout_probs_diff_std:0.00945594534277916 - training/rollout_actor_probs_pearson_corr:0.9991330504417419 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.000195860862732 - rollout_corr/rollout_is_ratio_fraction_high:9.644503734307364e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.9289007468614727e-05 - rollout_corr/rollout_is_max:2.3271644115448 - rollout_corr/rollout_is_min:0.296479195356369 - rollout_corr/rollout_is_std:0.029066067188978195 - rollout_corr/rollout_is_eff_sample_size:0.9991562339352135 - rollout_corr/rollout_is_seq_mean:1.0002284049987793 - rollout_corr/rollout_is_seq_std:0.0022042470518499613 - rollout_corr/rollout_is_seq_max:1.0126055479049683 - rollout_corr/rollout_is_seq_min:0.9936482906341553 - rollout_corr/rollout_is_seq_max_deviation:0.012605547904968262 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.261983500327915) - rollout_corr/training_log_ppl:np.float64(0.21683665785531048) - rollout_corr/kl:np.float64(0.0002809870581792495) - rollout_corr/k3_kl:np.float64(0.00047995293701319497) - rollout_corr/rollout_ppl:np.float64(1.261621960438788) - rollout_corr/rollout_log_ppl:np.float64(0.2165556696272688) - rollout_corr/log_ppl_diff:np.float64(0.00028098822804167867) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015070391818881035) - rollout_corr/log_ppl_diff_max:np.float64(0.007703110575675964) - rollout_corr/log_ppl_diff_min:np.float64(-0.010310009121894836) - rollout_corr/ppl_ratio:np.float64(1.0002832827158272) - rollout_corr/chi2_token:np.float64(0.0013595439959317446) - rollout_corr/chi2_seq:np.float64(0.5350547896232456) - actor/pg_loss:np.float64(0.001982162822969258) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023478754496863985) - actor/ppo_kl:np.float64(4.076835085786712e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(17187.49791721138) - self_distillation/token_reweight_w_std:np.float64(332695.3507974972) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003862450830638) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05358828179305419) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.1963564045727253) - perf/mfu/actor:np.float64(0.05060214812294124) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.50678253173828) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010506480932235718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010506480932235718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:405.0234375 - response_length/max:1677.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:405.0234375 - response_length_non_aborted/max:1677.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001492369920015335 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9843822922557592) - timing_s/agent_loop/generate_sequences/max:np.float64(12.267383886501193) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.2397227422043215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.267383886501193) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:1677 - timing_s/gen:19.05756193958223 - timing_s/reward:0.05966056138277054 - timing_s/old_log_prob:3.105440441519022 - timing_s/adv:0.6150863282382488 - timing_s/update_actor:21.63486903347075 - timing_s/step:44.565065210685134 - timing_s/stop_profile:0.00013065338134765625 - timing_per_token_ms/gen:0.1838007246839711 - timing_per_token_ms/update_actor:0.1280669908572031 - timing_per_token_ms/adv:0.0036409859959407155 - perf/total_num_tokens:168934 - perf/time_per_step:44.565065210685134 - perf/throughput:473.840886357257 (TaskRunner pid=2320396) Training Progress: 32%|███▏ | 32/100 [32:09<1:12:12, 63.71s/it] (TaskRunner pid=2320396) step:33 - global_seqlen/min:20278 - global_seqlen/max:32793 - global_seqlen/minmax_diff:12515 - global_seqlen/balanced_min:25854 - global_seqlen/balanced_max:25862 - global_seqlen/mean:25858.5 - actor/entropy:0.2247491031885147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24133732914924622 - training/rollout_probs_diff_mean:0.0028703457210212946 - training/rollout_probs_diff_std:0.008939353749155998 - training/rollout_actor_probs_pearson_corr:0.9992457628250122 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:0.9999353885650635 - rollout_corr/rollout_is_ratio_fraction_high:7.54580287321005e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.54580287321005e-06 - rollout_corr/rollout_is_max:2.0470011234283447 - rollout_corr/rollout_is_min:0.4843989908695221 - rollout_corr/rollout_is_std:0.02728046104311943 - rollout_corr/rollout_is_eff_sample_size:0.9992563298621977 - rollout_corr/rollout_is_seq_mean:0.9998895525932312 - rollout_corr/rollout_is_seq_std:0.001413932885043323 - rollout_corr/rollout_is_seq_max:1.0037660598754883 - rollout_corr/rollout_is_seq_min:0.9910454750061035 - rollout_corr/rollout_is_seq_max_deviation:0.008954524993896484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2669287519529462) - rollout_corr/training_log_ppl:np.float64(0.22308755599078722) - rollout_corr/kl:np.float64(0.0005338981650639951) - rollout_corr/k3_kl:np.float64(0.0004518400677682166) - rollout_corr/rollout_ppl:np.float64(1.2662203731015325) - rollout_corr/rollout_log_ppl:np.float64(0.22255365665478166) - rollout_corr/log_ppl_diff:np.float64(0.000533899336005561) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012277777277631685) - rollout_corr/log_ppl_diff_max:np.float64(0.01045304536819458) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037618353962898254) - rollout_corr/ppl_ratio:np.float64(1.0005353440064937) - rollout_corr/chi2_token:np.float64(0.0007296854164451361) - rollout_corr/chi2_seq:np.float64(0.41097239311784506) - actor/pg_loss:np.float64(0.0023052702890709043) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027347075024408696) - actor/ppo_kl:np.float64(1.4708585448275358e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(3411.1585964083206) - self_distillation/token_reweight_w_std:np.float64(86631.71416301752) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000493316212669) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05668039701413363) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.19899238646030426) - perf/mfu/actor:np.float64(0.06038801604738149) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.45315170288086) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006789336446672678 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006789336446672678 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:517.671875 - response_length/max:1329.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:517.671875 - response_length_non_aborted/max:1329.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013659894466400146 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2961205057799816) - timing_s/agent_loop/generate_sequences/max:np.float64(11.317749934270978) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.581255443590635) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.317749934270978) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1329 - timing_s/gen:18.672686226665974 - timing_s/reward:0.06610148772597313 - timing_s/old_log_prob:3.063822403550148 - timing_s/adv:0.623656140640378 - timing_s/update_actor:22.36642457358539 - timing_s/step:44.8817024435848 - timing_s/stop_profile:0.00011933408677577972 - timing_per_token_ms/gen:0.14090041220206131 - timing_per_token_ms/update_actor:0.1081193059032107 - timing_per_token_ms/adv:0.0030147540491539436 - perf/total_num_tokens:206868 - perf/time_per_step:44.8817024435848 - perf/throughput:576.1479309414232 (TaskRunner pid=2320396) Training Progress: 33%|███▎ | 33/100 [32:54<1:04:51, 58.08s/it] (TaskRunner pid=2320396) step:34 - global_seqlen/min:18809 - global_seqlen/max:30541 - global_seqlen/minmax_diff:11732 - global_seqlen/balanced_min:22984 - global_seqlen/balanced_max:23001 - global_seqlen/mean:22996.375 - actor/entropy:0.2210918515920639 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4279317259788513 - training/rollout_probs_diff_mean:0.002838705899193883 - training/rollout_probs_diff_std:0.009284661151468754 - training/rollout_actor_probs_pearson_corr:0.9991373419761658 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0000059604644775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.49451680551283e-06 - rollout_corr/rollout_is_max:1.964918613433838 - rollout_corr/rollout_is_min:0.022180255502462387 - rollout_corr/rollout_is_std:0.027658019214868546 - rollout_corr/rollout_is_eff_sample_size:0.9992357377417861 - rollout_corr/rollout_is_seq_mean:1.000097632408142 - rollout_corr/rollout_is_seq_std:0.0018793637864291668 - rollout_corr/rollout_is_seq_max:1.0091444253921509 - rollout_corr/rollout_is_seq_min:0.9919560551643372 - rollout_corr/rollout_is_seq_max_deviation:0.009144425392150879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2141821277327836) - rollout_corr/training_log_ppl:np.float64(0.18632135271036532) - rollout_corr/kl:np.float64(0.00037837233885004706) - rollout_corr/k3_kl:np.float64(0.0005182297692556403) - rollout_corr/rollout_ppl:np.float64(1.2136791995726526) - rollout_corr/rollout_log_ppl:np.float64(0.18594297835079487) - rollout_corr/log_ppl_diff:np.float64(0.00037837435957044363) - rollout_corr/log_ppl_abs_diff:np.float64(0.001449274888727814) - rollout_corr/log_ppl_diff_max:np.float64(0.010157078504562378) - rollout_corr/log_ppl_diff_min:np.float64(-0.007487893104553223) - rollout_corr/ppl_ratio:np.float64(1.0003805679734796) - rollout_corr/chi2_token:np.float64(0.0012797676026821136) - rollout_corr/chi2_seq:np.float64(0.4169660920742899) - actor/pg_loss:np.float64(0.0021550224628299475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045285703208719497) - actor/ppo_kl:np.float64(4.1904223541777696e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(5599.005817859666) - self_distillation/token_reweight_w_std:np.float64(112004.81754589331) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004518062341958) - self_distillation/token_reweight_w_clip_frac:np.float64(0.057579226355301216) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2537267468869686) - perf/mfu/actor:np.float64(0.05430294950212631) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.41295623779297) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006035353988409042 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006035353988409042 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.85546875 - response_length/max:1588.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.85546875 - response_length_non_aborted/max:1588.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012004747986793518 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2501347120851278) - timing_s/agent_loop/generate_sequences/max:np.float64(12.419921029359102) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.819320628768764) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.419921029359102) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1588 - timing_s/gen:19.884517431259155 - timing_s/reward:0.06309028342366219 - timing_s/old_log_prob:3.156838323920965 - timing_s/adv:0.5906004216521978 - timing_s/update_actor:22.091687720268965 - timing_s/step:45.87465010210872 - timing_s/stop_profile:0.00013749301433563232 - timing_per_token_ms/gen:0.1689093671691951 - timing_per_token_ms/update_actor:0.1200824462565783 - timing_per_token_ms/adv:0.003210290870040375 - perf/total_num_tokens:183971 - perf/time_per_step:45.87465010210872 - perf/throughput:501.2872021653398 (TaskRunner pid=2320396) Training Progress: 34%|███▍ | 34/100 [33:39<59:52, 54.43s/it] (TaskRunner pid=2320396) step:35 - global_seqlen/min:18781 - global_seqlen/max:27491 - global_seqlen/minmax_diff:8710 - global_seqlen/balanced_min:22623 - global_seqlen/balanced_max:22626 - global_seqlen/mean:22624.75 - actor/entropy:0.23807327449321747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18412302434444427 - training/rollout_probs_diff_mean:0.003115108236670494 - training/rollout_probs_diff_std:0.00949728861451149 - training/rollout_actor_probs_pearson_corr:0.999127209186554 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9999211430549622 - rollout_corr/rollout_is_ratio_fraction_high:1.743040957080666e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6145613446715288e-05 - rollout_corr/rollout_is_max:2.6255035400390625 - rollout_corr/rollout_is_min:0.4158304035663605 - rollout_corr/rollout_is_std:0.0292867012321949 - rollout_corr/rollout_is_eff_sample_size:0.9991427861890552 - rollout_corr/rollout_is_seq_mean:0.9998403787612915 - rollout_corr/rollout_is_seq_std:0.001774416770786047 - rollout_corr/rollout_is_seq_max:1.0043466091156006 - rollout_corr/rollout_is_seq_min:0.9918447732925415 - rollout_corr/rollout_is_seq_max_deviation:0.008155226707458496 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.265632627531886) - rollout_corr/training_log_ppl:np.float64(0.22240729215263855) - rollout_corr/kl:np.float64(0.000700322022321842) - rollout_corr/k3_kl:np.float64(0.0004940029620996711) - rollout_corr/rollout_ppl:np.float64(1.2647189851850271) - rollout_corr/rollout_log_ppl:np.float64(0.22170696874673013) - rollout_corr/log_ppl_diff:np.float64(0.0007003234059084207) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015176253218669444) - rollout_corr/log_ppl_diff_max:np.float64(0.008863285183906555) - rollout_corr/log_ppl_diff_min:np.float64(-0.003641173243522644) - rollout_corr/ppl_ratio:np.float64(1.0007026130333543) - rollout_corr/chi2_token:np.float64(0.0005808144342154264) - rollout_corr/chi2_seq:np.float64(0.4950171331875026) - actor/pg_loss:np.float64(0.0020264311460778117) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021731189508500393) - actor/ppo_kl:np.float64(7.550312483939337e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(530.5515484747011) - self_distillation/token_reweight_w_std:np.float64(11889.211997900158) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006914176046848) - self_distillation/token_reweight_w_clip_frac:np.float64(0.046933838588302024) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.15660720132291317) - perf/mfu/actor:np.float64(0.0544778590588236) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.24555206298828) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003924021031707525 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003924021031707525 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:448.2109375 - response_length/max:1032.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.2109375 - response_length_non_aborted/max:1032.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000113682821393013 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9868113528937101) - timing_s/agent_loop/generate_sequences/max:np.float64(9.072009176015854) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.959669176765601) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.072009176015854) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:994 - timing_s/gen:15.984084367752075 - timing_s/reward:0.06343846768140793 - timing_s/old_log_prob:2.9839373845607042 - timing_s/adv:0.5477243233472109 - timing_s/update_actor:21.675551645457745 - timing_s/step:41.344858787953854 - timing_s/stop_profile:0.00011985935270786285 - timing_per_token_ms/gen:0.13930456474309386 - timing_per_token_ms/update_actor:0.11975575224841017 - timing_per_token_ms/adv:0.003026134671914667 - perf/total_num_tokens:180998 - perf/time_per_step:41.344858787953854 - perf/throughput:547.2203960360822 (TaskRunner pid=2320396) Training Progress: 35%|███▌ | 35/100 [34:21<54:43, 50.52s/it] (TaskRunner pid=2320396) step:36 - global_seqlen/min:20139 - global_seqlen/max:31367 - global_seqlen/minmax_diff:11228 - global_seqlen/balanced_min:27336 - global_seqlen/balanced_max:27346 - global_seqlen/mean:27343.75 - actor/entropy:0.2504202723503113 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40465080738067627 - training/rollout_probs_diff_mean:0.0030166946817189455 - training/rollout_probs_diff_std:0.009248757734894753 - training/rollout_actor_probs_pearson_corr:0.9992465972900391 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8515625 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8515625 - rollout_corr/rollout_is_mean:0.9999235272407532 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0037670765304938e-05 - rollout_corr/rollout_is_max:1.5995163917541504 - rollout_corr/rollout_is_min:0.43956276774406433 - rollout_corr/rollout_is_std:0.028227554634213448 - rollout_corr/rollout_is_eff_sample_size:0.9992037204657419 - rollout_corr/rollout_is_seq_mean:0.9999057054519653 - rollout_corr/rollout_is_seq_std:0.0012788664316758513 - rollout_corr/rollout_is_seq_max:1.0031697750091553 - rollout_corr/rollout_is_seq_min:0.9950219392776489 - rollout_corr/rollout_is_seq_max_deviation:0.004978060722351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2692863936536014) - rollout_corr/training_log_ppl:np.float64(0.22850381008174736) - rollout_corr/kl:np.float64(0.0005316972360809302) - rollout_corr/k3_kl:np.float64(0.0004293374125552418) - rollout_corr/rollout_ppl:np.float64(1.2685975637286901) - rollout_corr/rollout_log_ppl:np.float64(0.22797211322176736) - rollout_corr/log_ppl_diff:np.float64(0.000531696859980002) - rollout_corr/log_ppl_abs_diff:np.float64(0.001130868069594726) - rollout_corr/log_ppl_diff_max:np.float64(0.008397668600082397) - rollout_corr/log_ppl_diff_min:np.float64(-0.0027961134910583496) - rollout_corr/ppl_ratio:np.float64(1.000532856443897) - rollout_corr/chi2_token:np.float64(0.0006337745580822229) - rollout_corr/chi2_seq:np.float64(0.33934940421022475) - actor/pg_loss:np.float64(0.0029051536694169044) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002513188455850468) - actor/ppo_kl:np.float64(3.129442966631757e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(6897.516162533779) - self_distillation/token_reweight_w_std:np.float64(169786.0625592887) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015529317315668) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07552138640312478) - self_distillation/empty_target_batch:np.float64(0.1484375) - actor/grad_norm:np.float64(0.1749717816710472) - perf/mfu/actor:np.float64(0.06304641565475115) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.2574691772461) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006652506534010172 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006652506534010172 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:584.8359375 - response_length/max:1433.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:584.8359375 - response_length_non_aborted/max:1433.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.780377149581909e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0670874547213316) - timing_s/agent_loop/generate_sequences/max:np.float64(12.71995124220848) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.654472718997567) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.71995124220848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:1433 - timing_s/gen:19.51580721512437 - timing_s/reward:0.0687258280813694 - timing_s/old_log_prob:3.301211541518569 - timing_s/adv:0.6224024724215269 - timing_s/update_actor:22.644038517028093 - timing_s/step:46.24068879336119 - timing_s/stop_profile:0.00012005679309368134 - timing_per_token_ms/gen:0.13035044026185474 - timing_per_token_ms/update_actor:0.10351560464927129 - timing_per_token_ms/adv:0.0028452684453555517 - perf/total_num_tokens:218750 - perf/time_per_step:46.24068879336119 - perf/throughput:591.3352658346596 (TaskRunner pid=2320396) Training Progress: 36%|███▌ | 36/100 [35:07<52:32, 49.25s/it] (TaskRunner pid=2320396) step:37 - global_seqlen/min:20086 - global_seqlen/max:24803 - global_seqlen/minmax_diff:4717 - global_seqlen/balanced_min:23730 - global_seqlen/balanced_max:23741 - global_seqlen/mean:23735.75 - actor/entropy:0.23037061095237732 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6908016204833984 - training/rollout_probs_diff_mean:0.002985295606777072 - training/rollout_probs_diff_std:0.009590771049261093 - training/rollout_actor_probs_pearson_corr:0.9990717768669128 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000369548797607 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.05903447244782e-05 - rollout_corr/rollout_is_max:1.6410112380981445 - rollout_corr/rollout_is_min:0.2311713546514511 - rollout_corr/rollout_is_std:0.028421156108379364 - rollout_corr/rollout_is_eff_sample_size:0.9991930088240167 - rollout_corr/rollout_is_seq_mean:0.9999706745147705 - rollout_corr/rollout_is_seq_std:0.0017531486228108406 - rollout_corr/rollout_is_seq_max:1.00639009475708 - rollout_corr/rollout_is_seq_min:0.9936760663986206 - rollout_corr/rollout_is_seq_max_deviation:0.006390094757080078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2430013921111822) - rollout_corr/training_log_ppl:np.float64(0.20931774778000545) - rollout_corr/kl:np.float64(0.0005845249421625454) - rollout_corr/k3_kl:np.float64(0.0004938606622033603) - rollout_corr/rollout_ppl:np.float64(1.242268514353782) - rollout_corr/rollout_log_ppl:np.float64(0.20873322064289823) - rollout_corr/log_ppl_diff:np.float64(0.0005845271371072158) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014318034081952646) - rollout_corr/log_ppl_diff_max:np.float64(0.008943051099777222) - rollout_corr/log_ppl_diff_min:np.float64(-0.009509354829788208) - rollout_corr/ppl_ratio:np.float64(1.0005866941064596) - rollout_corr/chi2_token:np.float64(0.0007902453653514385) - rollout_corr/chi2_seq:np.float64(0.5003544613718987) - actor/pg_loss:np.float64(0.002713275025598705) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003606915511227271) - actor/ppo_kl:np.float64(7.810856640055874e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(4877.020437426167) - self_distillation/token_reweight_w_std:np.float64(107261.42743945995) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000146112870425) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05535905514261685) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.17725222185254097) - perf/mfu/actor:np.float64(0.055516710245582246) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.18002700805664) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007146945223212242 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.007146945223212242 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:481.1796875 - response_length/max:1472.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:481.1796875 - response_length_non_aborted/max:1472.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.340047836303711e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.017892599105835) - timing_s/agent_loop/generate_sequences/max:np.float64(11.753218919038773) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.032719595379604) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.753218919038773) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:1472 - timing_s/gen:18.823624592274427 - timing_s/reward:0.0637381449341774 - timing_s/old_log_prob:3.0538349244743586 - timing_s/adv:0.6095426809042692 - timing_s/update_actor:22.17010228894651 - timing_s/step:44.80895793251693 - timing_s/stop_profile:0.0001234728842973709 - timing_per_token_ms/gen:0.15281148700519903 - timing_per_token_ms/update_actor:0.11675480177025431 - timing_per_token_ms/adv:0.0032100454004206167 - perf/total_num_tokens:189886 - perf/time_per_step:44.80895793251693 - perf/throughput:529.7099306738276 (TaskRunner pid=2320396) Training Progress: 37%|███▋ | 37/100 [35:52<50:19, 47.93s/it] (TaskRunner pid=2320396) step:38 - global_seqlen/min:19691 - global_seqlen/max:29519 - global_seqlen/minmax_diff:9828 - global_seqlen/balanced_min:23230 - global_seqlen/balanced_max:23236 - global_seqlen/mean:23233.125 - actor/entropy:0.22175385057926178 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424724340438843 - training/rollout_probs_diff_mean:0.002874217461794615 - training/rollout_probs_diff_std:0.00937693938612938 - training/rollout_actor_probs_pearson_corr:0.999193012714386 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999099969863892 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8694218397140503 - rollout_corr/rollout_is_min:0.5457053184509277 - rollout_corr/rollout_is_std:0.02795809879899025 - rollout_corr/rollout_is_eff_sample_size:0.9992187171499983 - rollout_corr/rollout_is_seq_mean:0.9998998045921326 - rollout_corr/rollout_is_seq_std:0.00174828025046736 - rollout_corr/rollout_is_seq_max:1.0071884393692017 - rollout_corr/rollout_is_seq_min:0.9907048940658569 - rollout_corr/rollout_is_seq_max_deviation:0.009295105934143066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2191084562800825) - rollout_corr/training_log_ppl:np.float64(0.18708222404529806) - rollout_corr/kl:np.float64(0.0005426743667129585) - rollout_corr/k3_kl:np.float64(0.00042263647131335347) - rollout_corr/rollout_ppl:np.float64(1.2184375426732004) - rollout_corr/rollout_log_ppl:np.float64(0.1865395491477102) - rollout_corr/log_ppl_diff:np.float64(0.0005426748975878581) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012689346476690844) - rollout_corr/log_ppl_diff_max:np.float64(0.010417088866233826) - rollout_corr/log_ppl_diff_min:np.float64(-0.005928635597229004) - rollout_corr/ppl_ratio:np.float64(1.0005444309208542) - rollout_corr/chi2_token:np.float64(0.0006026304326951504) - rollout_corr/chi2_seq:np.float64(1.5354293449781835) - actor/pg_loss:np.float64(0.0027565507916733623) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002350132526771631) - actor/ppo_kl:np.float64(3.495901299288562e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(6817.583530688891) - self_distillation/token_reweight_w_std:np.float64(138689.49364839046) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017913763877004) - self_distillation/token_reweight_w_clip_frac:np.float64(0.056382376729743555) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1824763622134924) - perf/mfu/actor:np.float64(0.05564749761275871) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.17073440551758) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007229797076433897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007229797076433897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.25390625 - response_length/max:1222.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.25390625 - response_length_non_aborted/max:1222.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010543875396251678 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1841194070875645) - timing_s/agent_loop/generate_sequences/max:np.float64(10.229186763986945) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.03015022159525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.229186763986945) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:1222 - timing_s/gen:16.99055285193026 - timing_s/reward:0.06129678152501583 - timing_s/old_log_prob:3.0377829782664776 - timing_s/adv:0.5637715607881546 - timing_s/update_actor:21.657408459112048 - timing_s/step:42.39016137458384 - timing_s/stop_profile:0.00011087954044342041 - timing_per_token_ms/gen:0.14451558533227518 - timing_per_token_ms/update_actor:0.11652225249031312 - timing_per_token_ms/adv:0.0030332314356557428 - perf/total_num_tokens:185865 - perf/time_per_step:42.39016137458384 - perf/throughput:548.0782390682298 (TaskRunner pid=2320396) Training Progress: 38%|███▊ | 38/100 [36:34<47:49, 46.28s/it] (TaskRunner pid=2320396) step:39 - global_seqlen/min:14393 - global_seqlen/max:25409 - global_seqlen/minmax_diff:11016 - global_seqlen/balanced_min:20262 - global_seqlen/balanced_max:20274 - global_seqlen/mean:20270.375 - actor/entropy:0.20056596398353577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28917062282562256 - training/rollout_probs_diff_mean:0.002990458393469453 - training/rollout_probs_diff_std:0.009769958443939686 - training/rollout_actor_probs_pearson_corr:0.9990027546882629 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.80859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.000001072883606 - rollout_corr/rollout_is_ratio_fraction_high:2.082053651975002e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.164107303950004e-05 - rollout_corr/rollout_is_max:2.3594627380371094 - rollout_corr/rollout_is_min:0.1703754961490631 - rollout_corr/rollout_is_std:0.02931111305952072 - rollout_corr/rollout_is_eff_sample_size:0.999141596140458 - rollout_corr/rollout_is_seq_mean:0.9999600052833557 - rollout_corr/rollout_is_seq_std:0.001872700871899724 - rollout_corr/rollout_is_seq_max:1.0076165199279785 - rollout_corr/rollout_is_seq_min:0.993287205696106 - rollout_corr/rollout_is_seq_max_deviation:0.007616519927978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2189809894189239) - rollout_corr/training_log_ppl:np.float64(0.18452461713604862) - rollout_corr/kl:np.float64(0.0004751513001508556) - rollout_corr/k3_kl:np.float64(0.00048812701186662366) - rollout_corr/rollout_ppl:np.float64(1.2183574447408319) - rollout_corr/rollout_log_ppl:np.float64(0.1840494627977023) - rollout_corr/log_ppl_diff:np.float64(0.0004751543383463286) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013876069351681508) - rollout_corr/log_ppl_diff_max:np.float64(0.01167362928390503) - rollout_corr/log_ppl_diff_min:np.float64(-0.006826907396316528) - rollout_corr/ppl_ratio:np.float64(1.0004770983941853) - rollout_corr/chi2_token:np.float64(0.0010034546721726656) - rollout_corr/chi2_seq:np.float64(0.23432688671164215) - actor/pg_loss:np.float64(0.001627549878321588) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016317091376549797) - actor/ppo_kl:np.float64(-2.4777720572188855e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_w_mean:np.float64(10545.353267262224) - self_distillation/token_reweight_w_std:np.float64(130564.52004474157) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002583481837064) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03637115124729462) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.1302459230646491) - perf/mfu/actor:np.float64(0.04886280426766004) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.26232528686523) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027457084506750107 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027457084506750107 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:375.23046875 - response_length/max:1107.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:375.23046875 - response_length_non_aborted/max:1107.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499038189649582 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0094370376318693) - timing_s/agent_loop/generate_sequences/max:np.float64(8.979632189497352) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.206847889901837) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.979632189497352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:1107 - timing_s/gen:16.14283774420619 - timing_s/reward:0.05978567153215408 - timing_s/old_log_prob:3.030688526108861 - timing_s/adv:0.5634818132966757 - timing_s/update_actor:21.313188266009092 - timing_s/step:41.19874492101371 - timing_s/stop_profile:0.00010902248322963715 - timing_per_token_ms/gen:0.1680512783206799 - timing_per_token_ms/update_actor:0.1314306485820384 - timing_per_token_ms/adv:0.003474786562265595 - perf/total_num_tokens:162163 - perf/time_per_step:41.19874492101371 - perf/throughput:492.01438147842583 (TaskRunner pid=2320396) Training Progress: 39%|███▉ | 39/100 [37:16<45:31, 44.78s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) We solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) This is closest to option B: 56 pC. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2320396) step:40 - global_seqlen/min:17964 - global_seqlen/max:33806 - global_seqlen/minmax_diff:15842 - global_seqlen/balanced_min:23957 - global_seqlen/balanced_max:24000 - global_seqlen/mean:23991.125 - actor/entropy:0.17409266531467438 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22586679458618164 - training/rollout_probs_diff_mean:0.002714834176003933 - training/rollout_probs_diff_std:0.009270383976399899 - training/rollout_actor_probs_pearson_corr:0.9989473819732666 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.77734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:1.0000334978103638 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.398139076831285e-05 - rollout_corr/rollout_is_max:1.8032424449920654 - rollout_corr/rollout_is_min:0.3792487382888794 - rollout_corr/rollout_is_std:0.027413414791226387 - rollout_corr/rollout_is_eff_sample_size:0.9992491879902002 - rollout_corr/rollout_is_seq_mean:0.9999297857284546 - rollout_corr/rollout_is_seq_std:0.0016750209033489227 - rollout_corr/rollout_is_seq_max:1.0049023628234863 - rollout_corr/rollout_is_seq_min:0.9892888069152832 - rollout_corr/rollout_is_seq_max_deviation:0.010711193084716797 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1875572656281292) - rollout_corr/training_log_ppl:np.float64(0.16661910359107424) - rollout_corr/kl:np.float64(0.0007166074137661127) - rollout_corr/k3_kl:np.float64(0.0006648261616248874) - rollout_corr/rollout_ppl:np.float64(1.1866890429519117) - rollout_corr/rollout_log_ppl:np.float64(0.16590249516593758) - rollout_corr/log_ppl_diff:np.float64(0.0007166084251366556) - rollout_corr/log_ppl_abs_diff:np.float64(0.001578556781169027) - rollout_corr/log_ppl_diff_max:np.float64(0.07116454839706421) - rollout_corr/log_ppl_diff_min:np.float64(-0.004071809351444244) - rollout_corr/ppl_ratio:np.float64(1.0007289675995708) - rollout_corr/chi2_token:np.float64(0.000788895646110177) - rollout_corr/chi2_seq:np.float64(1.0485429209657013) - actor/pg_loss:np.float64(0.0022394302068278193) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032143645762516826) - actor/ppo_kl:np.float64(0.0002396634859533897) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_w_mean:np.float64(14596.109324033838) - self_distillation/token_reweight_w_std:np.float64(170215.50741613266) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997718355152756) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045137047200114466) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2175825722515583) - perf/mfu/actor:np.float64(0.056676871566027895) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.3233880996704) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.61796875) - val-aux/sciknoweval/reward/std@16:np.float64(0.20523842662574854) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.705825) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18368841387292015) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5308499999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16680965099369396) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.618025) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2043784429066801) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.783475) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13910305561491254) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46548750000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10917650565978679) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6298874999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1789536010234551) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8374375000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08838687383578706) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42711249999999995) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06170735155075495) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6369999999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14868065619864407) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8685500000000002) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04271179547353976) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4054375) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03508296536724832) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6418125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11962085895290062) - val-aux/sciknoweval/score/mean@16:np.float64(0.61796875) - val-aux/sciknoweval/score/std@16:np.float64(0.20523842662574854) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.705825) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18368841387292015) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5308499999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16680965099369396) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.618025) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2043784429066801) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.783475) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13910305561491254) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46548750000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10917650565978679) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6298874999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1789536010234551) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8374375000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08838687383578706) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42711249999999995) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06170735155075495) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6369999999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14868065619864407) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8685500000000002) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04271179547353976) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4054375) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03508296536724832) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6418125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11962085895290062) - val-core/sciknoweval/acc/mean@16:np.float64(0.61796875) - val-aux/sciknoweval/acc/std@16:np.float64(0.20523842662574854) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.705825) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18368841387292015) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5308499999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16680965099369396) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.618025) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2043784429066801) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.783475) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13910305561491254) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46548750000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10917650565978679) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6298874999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1789536010234551) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8374375000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08838687383578706) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42711249999999995) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06170735155075495) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6369999999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14868065619864407) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8685500000000002) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04271179547353976) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4054375) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03508296536724832) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6418125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11962085895290062) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.017180441164697836) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9962500000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.009611894943540346) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9837250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.018769314003885828) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9903625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.016320258609184714) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9991875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003454498723887649) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9764875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.019219229161406868) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.012089324810506192) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.000788986691902975) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9676874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.017749891876663433) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.996725) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008829929148801091) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9596875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.014249930113123078) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9987) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0038157568056677834) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009700472466647625 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009700472466647625 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:488.66015625 - response_length/max:1799.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:488.66015625 - response_length_non_aborted/max:1799.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014634616672992706 - timing_s/agent_loop/generate_sequences/min:np.float64(1.002776077017188) - timing_s/agent_loop/generate_sequences/max:np.float64(13.396277887746692) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.303398293312057) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.396277887746692) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1799 - timing_s/gen:20.2838581148535 - timing_s/reward:0.06532224267721176 - timing_s/old_log_prob:3.0899851340800524 - timing_s/adv:0.610138576477766 - timing_s/update_actor:21.942959178239107 - timing_s/step:46.080426421016455 - timing_s/testing:112.31239799968898 - timing_s/save_checkpoint:12.967904256656766 - timing_s/stop_profile:0.0001502949744462967 - timing_per_token_ms/gen:0.16214504036750282 - timing_per_token_ms/update_actor:0.1143285234552314 - timing_per_token_ms/adv:0.0031789806463732215 - perf/total_num_tokens:191929 - perf/time_per_step:46.080426421016455 - perf/throughput:520.6359155795069 (TaskRunner pid=2320396) Training Progress: 40%|████ | 40/100 [40:07<1:22:46, 82.78s/it] (TaskRunner pid=2320396) step:41 - global_seqlen/min:21172 - global_seqlen/max:28257 - global_seqlen/minmax_diff:7085 - global_seqlen/balanced_min:24104 - global_seqlen/balanced_max:24111 - global_seqlen/mean:24108.25 - actor/entropy:0.19510897994041443 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34874776005744934 - training/rollout_probs_diff_mean:0.00296596996486187 - training/rollout_probs_diff_std:0.009650228545069695 - training/rollout_actor_probs_pearson_corr:0.999030590057373 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.7734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:1.0000007152557373 - rollout_corr/rollout_is_ratio_fraction_high:8.208967301470693e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.208967301470693e-06 - rollout_corr/rollout_is_max:3.2528834342956543 - rollout_corr/rollout_is_min:0.25628554821014404 - rollout_corr/rollout_is_std:0.02881893701851368 - rollout_corr/rollout_is_eff_sample_size:0.9991700390777238 - rollout_corr/rollout_is_seq_mean:0.9999620914459229 - rollout_corr/rollout_is_seq_std:0.001639653230085969 - rollout_corr/rollout_is_seq_max:1.0059510469436646 - rollout_corr/rollout_is_seq_min:0.99149489402771 - rollout_corr/rollout_is_seq_max_deviation:0.008505105972290039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.217791025992483) - rollout_corr/training_log_ppl:np.float64(0.18964533759572078) - rollout_corr/kl:np.float64(0.0005734493415801012) - rollout_corr/k3_kl:np.float64(0.00044943573641376133) - rollout_corr/rollout_ppl:np.float64(1.2171180993318558) - rollout_corr/rollout_log_ppl:np.float64(0.18907188744924497) - rollout_corr/log_ppl_diff:np.float64(0.0005734501464758068) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012757474614772946) - rollout_corr/log_ppl_diff_max:np.float64(0.010674156248569489) - rollout_corr/log_ppl_diff_min:np.float64(-0.005041897296905518) - rollout_corr/ppl_ratio:np.float64(1.0005749089177698) - rollout_corr/chi2_token:np.float64(0.0006430486682802439) - rollout_corr/chi2_seq:np.float64(0.34916975046508014) - actor/pg_loss:np.float64(0.0028609149158000946) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00037430567363117007) - actor/ppo_kl:np.float64(0.00010280042299903869) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_w_mean:np.float64(15439.003869137494) - self_distillation/token_reweight_w_std:np.float64(381955.13341238984) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000298652332276) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05494706741592381) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.19181200489401817) - perf/mfu/actor:np.float64(0.05644916039033881) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.62611770629883) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020902082324028015 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020902082324028015 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.8515625 - response_length/max:1251.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:475.8515625 - response_length_non_aborted/max:1251.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010300613939762115 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0740045364946127) - timing_s/agent_loop/generate_sequences/max:np.float64(10.960649330168962) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.377686426785658) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.960649330168962) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1251 - timing_s/gen:17.190705524757504 - timing_s/reward:0.06419655494391918 - timing_s/old_log_prob:3.0940296556800604 - timing_s/adv:0.6143710687756538 - timing_s/update_actor:22.368526071310043 - timing_s/step:43.42171490751207 - timing_s/stop_profile:0.00011404231190681458 - timing_per_token_ms/gen:0.141117942543446 - timing_per_token_ms/update_actor:0.11597962352778636 - timing_per_token_ms/adv:0.0031854814678359787 - perf/total_num_tokens:192866 - perf/time_per_step:43.42171490751207 - perf/throughput:555.2118347087487 (TaskRunner pid=2320396) Training Progress: 41%|████ | 41/100 [40:51<1:09:48, 70.99s/it] (TaskRunner pid=2320396) step:42 - global_seqlen/min:22569 - global_seqlen/max:27311 - global_seqlen/minmax_diff:4742 - global_seqlen/balanced_min:25038 - global_seqlen/balanced_max:25054 - global_seqlen/mean:25047.0 - actor/entropy:0.24892261624336243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.670967161655426 - training/rollout_probs_diff_mean:0.003025430254638195 - training/rollout_probs_diff_std:0.009795612655580044 - training/rollout_actor_probs_pearson_corr:0.9991313219070435 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998930096626282 - rollout_corr/rollout_is_ratio_fraction_high:1.5133171473280527e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.783293141168542e-05 - rollout_corr/rollout_is_max:4.227712631225586 - rollout_corr/rollout_is_min:0.18747776746749878 - rollout_corr/rollout_is_std:0.029035290703177452 - rollout_corr/rollout_is_eff_sample_size:0.9991574240186801 - rollout_corr/rollout_is_seq_mean:0.9998325109481812 - rollout_corr/rollout_is_seq_std:0.0015463787131011486 - rollout_corr/rollout_is_seq_max:1.0083459615707397 - rollout_corr/rollout_is_seq_min:0.9951817393302917 - rollout_corr/rollout_is_seq_max_deviation:0.008345961570739746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2542899283580482) - rollout_corr/training_log_ppl:np.float64(0.2118087612761883) - rollout_corr/kl:np.float64(0.0006046219645430284) - rollout_corr/k3_kl:np.float64(0.0004428104627720586) - rollout_corr/rollout_ppl:np.float64(1.2534998757764697) - rollout_corr/rollout_log_ppl:np.float64(0.21120413720200304) - rollout_corr/log_ppl_diff:np.float64(0.0006046240741852671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012860278657171875) - rollout_corr/log_ppl_diff_max:np.float64(0.005592525005340576) - rollout_corr/log_ppl_diff_min:np.float64(-0.007412515580654144) - rollout_corr/ppl_ratio:np.float64(1.0006060795858502) - rollout_corr/chi2_token:np.float64(0.0005524898879230022) - rollout_corr/chi2_seq:np.float64(0.7299846305977553) - actor/pg_loss:np.float64(0.004190075560472906) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002905324188304803) - actor/ppo_kl:np.float64(3.560879972663855e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(3159.014352656668) - self_distillation/token_reweight_w_std:np.float64(83420.70022474515) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017929573077708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06845499108021613) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.227945476770401) - perf/mfu/actor:np.float64(0.057657091310341184) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.51451110839844) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014856046065688133 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014856046065688133 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:516.25 - response_length/max:1516.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:516.25 - response_length_non_aborted/max:1516.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010149739682674408 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0408725664019585) - timing_s/agent_loop/generate_sequences/max:np.float64(12.283551223576069) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.583720707065368) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.283551223576069) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1516 - timing_s/gen:18.965983686968684 - timing_s/reward:0.07659243419766426 - timing_s/old_log_prob:3.0534052439033985 - timing_s/adv:0.6341004334390163 - timing_s/update_actor:22.555470870807767 - timing_s/step:45.37643878906965 - timing_s/stop_profile:0.00011052936315536499 - timing_per_token_ms/gen:0.143507745815441 - timing_per_token_ms/update_actor:0.11256573078017211 - timing_per_token_ms/adv:0.0031645528079161994 - perf/total_num_tokens:200376 - perf/time_per_step:45.37643878906965 - perf/throughput:551.9824972697804 (TaskRunner pid=2320396) Training Progress: 42%|████▏ | 42/100 [41:36<1:01:12, 63.32s/it] (TaskRunner pid=2320396) step:43 - global_seqlen/min:19052 - global_seqlen/max:31919 - global_seqlen/minmax_diff:12867 - global_seqlen/balanced_min:24552 - global_seqlen/balanced_max:24917 - global_seqlen/mean:24612.5 - actor/entropy:0.23190994560718536 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5026711821556091 - training/rollout_probs_diff_mean:0.003001613076776266 - training/rollout_probs_diff_std:0.00961999874562025 - training/rollout_actor_probs_pearson_corr:0.9990864396095276 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999968409538269 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.564602462167386e-05 - rollout_corr/rollout_is_max:1.974767804145813 - rollout_corr/rollout_is_min:0.15729260444641113 - rollout_corr/rollout_is_std:0.02876925654709339 - rollout_corr/rollout_is_eff_sample_size:0.9991730143739226 - rollout_corr/rollout_is_seq_mean:1.0000637769699097 - rollout_corr/rollout_is_seq_std:0.001507977838627994 - rollout_corr/rollout_is_seq_max:1.007026195526123 - rollout_corr/rollout_is_seq_min:0.9949943423271179 - rollout_corr/rollout_is_seq_max_deviation:0.007026195526123047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2377484519965947) - rollout_corr/training_log_ppl:np.float64(0.2034379402466584) - rollout_corr/kl:np.float64(0.0003022583822458458) - rollout_corr/k3_kl:np.float64(0.0004391857282328715) - rollout_corr/rollout_ppl:np.float64(1.2373604252934456) - rollout_corr/rollout_log_ppl:np.float64(0.20313568067649612) - rollout_corr/log_ppl_diff:np.float64(0.00030225957016227767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011025687199435197) - rollout_corr/log_ppl_diff_max:np.float64(0.003672093152999878) - rollout_corr/log_ppl_diff_min:np.float64(-0.00567355751991272) - rollout_corr/ppl_ratio:np.float64(1.0003033550456166) - rollout_corr/chi2_token:np.float64(0.001162565080448985) - rollout_corr/chi2_seq:np.float64(0.39001374156214297) - actor/pg_loss:np.float64(0.004258588422089815) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024049899820965948) - actor/ppo_kl:np.float64(-5.5626800181229896e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(2705.9410314837005) - self_distillation/token_reweight_w_std:np.float64(74205.18795861094) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006196503527462) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06989276569220237) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.19171869941055775) - perf/mfu/actor:np.float64(0.05718318820002838) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.54368352890015) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008528061211109161 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008528061211109161 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:499.328125 - response_length/max:2008.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.328125 - response_length_non_aborted/max:2008.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013203918933868408 - timing_s/agent_loop/generate_sequences/min:np.float64(1.648325640708208) - timing_s/agent_loop/generate_sequences/max:np.float64(14.80245334841311) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.24944398264779) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.80245334841311) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:2008 - timing_s/gen:21.46511185541749 - timing_s/reward:0.06564498879015446 - timing_s/old_log_prob:3.178984245285392 - timing_s/adv:0.6894926764070988 - timing_s/update_actor:22.444915840402246 - timing_s/step:47.931692676618695 - timing_s/stop_profile:0.00011303648352622986 - timing_per_token_ms/gen:0.16792183133130056 - timing_per_token_ms/update_actor:0.11399144662469399 - timing_per_token_ms/adv:0.003501740357577952 - perf/total_num_tokens:196900 - perf/time_per_step:47.931692676618695 - perf/throughput:513.4911501259394 (TaskRunner pid=2320396) Training Progress: 43%|████▎ | 43/100 [42:24<55:46, 58.72s/it] (TaskRunner pid=2320396) step:44 - global_seqlen/min:19418 - global_seqlen/max:31085 - global_seqlen/minmax_diff:11667 - global_seqlen/balanced_min:25774 - global_seqlen/balanced_max:25804 - global_seqlen/mean:25789.125 - actor/entropy:0.23197732865810394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35145077109336853 - training/rollout_probs_diff_mean:0.002846738323569298 - training/rollout_probs_diff_std:0.008968722075223923 - training/rollout_actor_probs_pearson_corr:0.999229371547699 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000036597251892 - rollout_corr/rollout_is_ratio_fraction_high:7.387652203760808e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.387652203760808e-06 - rollout_corr/rollout_is_max:2.6363072395324707 - rollout_corr/rollout_is_min:0.4945003390312195 - rollout_corr/rollout_is_std:0.027107305824756622 - rollout_corr/rollout_is_eff_sample_size:0.9992657334826961 - rollout_corr/rollout_is_seq_mean:1.0000344514846802 - rollout_corr/rollout_is_seq_std:0.001453010132536292 - rollout_corr/rollout_is_seq_max:1.0084365606307983 - rollout_corr/rollout_is_seq_min:0.9961057901382446 - rollout_corr/rollout_is_seq_max_deviation:0.00843656063079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.260404644999653) - rollout_corr/training_log_ppl:np.float64(0.21625306174973957) - rollout_corr/kl:np.float64(0.00030714693509725777) - rollout_corr/k3_kl:np.float64(0.0004374984173551866) - rollout_corr/rollout_ppl:np.float64(1.260006621479988) - rollout_corr/rollout_log_ppl:np.float64(0.2159459152026102) - rollout_corr/log_ppl_diff:np.float64(0.0003071465471293777) - rollout_corr/log_ppl_abs_diff:np.float64(0.001081082969903946) - rollout_corr/log_ppl_diff_max:np.float64(0.004448056221008301) - rollout_corr/log_ppl_diff_min:np.float64(-0.0050467848777771) - rollout_corr/ppl_ratio:np.float64(1.0003082156181335) - rollout_corr/chi2_token:np.float64(0.0011717674788087606) - rollout_corr/chi2_seq:np.float64(0.48563542030751705) - actor/pg_loss:np.float64(0.002838829648680985) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029677772226932575) - actor/ppo_kl:np.float64(-4.0592053640153836e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(11010.470829747384) - self_distillation/token_reweight_w_std:np.float64(210548.85107363865) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000365384388715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06577898818068206) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.20856376737356186) - perf/mfu/actor:np.float64(0.05939696075292037) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.48828506469727) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020655876025557518 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020655876025557518 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:528.75390625 - response_length/max:1759.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:528.75390625 - response_length_non_aborted/max:1759.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.023956954479218e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3401075471192598) - timing_s/agent_loop/generate_sequences/max:np.float64(13.932623855769634) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.293072881489934) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.932623855769634) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:282 - timing_s/agent_loop/slowest/response_length:1759 - timing_s/gen:21.01163581199944 - timing_s/reward:0.06546073779463768 - timing_s/old_log_prob:3.0930407010018826 - timing_s/adv:0.6480591148138046 - timing_s/update_actor:22.687276219949126 - timing_s/step:47.591358166188 - timing_s/stop_profile:0.00011670403182506561 - timing_per_token_ms/gen:0.15522665917065803 - timing_per_token_ms/update_actor:0.10996532559726788 - timing_per_token_ms/adv:0.0031411453219807022 - perf/total_num_tokens:206313 - perf/time_per_step:47.591358166188 - perf/throughput:541.8867204828433 (TaskRunner pid=2320396) Training Progress: 44%|████▍ | 44/100 [43:12<51:41, 55.39s/it] (TaskRunner pid=2320396) step:45 - global_seqlen/min:19411 - global_seqlen/max:31128 - global_seqlen/minmax_diff:11717 - global_seqlen/balanced_min:24868 - global_seqlen/balanced_max:24877 - global_seqlen/mean:24872.0 - actor/entropy:0.2606814205646515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37384867668151855 - training/rollout_probs_diff_mean:0.0030074005480855703 - training/rollout_probs_diff_std:0.009297839365899563 - training/rollout_actor_probs_pearson_corr:0.9992638230323792 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.999846875667572 - rollout_corr/rollout_is_ratio_fraction_high:7.83306222729152e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.566612445458304e-05 - rollout_corr/rollout_is_max:2.1816329956054688 - rollout_corr/rollout_is_min:0.03358318656682968 - rollout_corr/rollout_is_std:0.02886233665049076 - rollout_corr/rollout_is_eff_sample_size:0.9991675398426103 - rollout_corr/rollout_is_seq_mean:0.9997819662094116 - rollout_corr/rollout_is_seq_std:0.0016762863378971815 - rollout_corr/rollout_is_seq_max:1.0064592361450195 - rollout_corr/rollout_is_seq_min:0.9929512143135071 - rollout_corr/rollout_is_seq_max_deviation:0.00704878568649292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3119843625463545) - rollout_corr/training_log_ppl:np.float64(0.2495773843547795) - rollout_corr/kl:np.float64(0.0005540147624278902) - rollout_corr/k3_kl:np.float64(0.0005072801855448006) - rollout_corr/rollout_ppl:np.float64(1.3112443177960813) - rollout_corr/rollout_log_ppl:np.float64(0.24902336824743543) - rollout_corr/log_ppl_diff:np.float64(0.0005540161073440686) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013045282248640433) - rollout_corr/log_ppl_diff_max:np.float64(0.0060444176197052) - rollout_corr/log_ppl_diff_min:np.float64(-0.005649633705615997) - rollout_corr/ppl_ratio:np.float64(1.0005556086543947) - rollout_corr/chi2_token:np.float64(0.0009730139281600714) - rollout_corr/chi2_seq:np.float64(1.1036222074180841) - actor/pg_loss:np.float64(0.0031190799782052636) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035551090218177706) - actor/ppo_kl:np.float64(-0.00010970474031912758) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(530.7625214133877) - self_distillation/token_reweight_w_std:np.float64(14348.925151058473) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011988517362624) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07025789209001232) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.22478391602635384) - perf/mfu/actor:np.float64(0.05839612980819302) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.5486536026001) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001738939667120576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001738939667120576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:498.6875 - response_length/max:1215.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:498.6875 - response_length_non_aborted/max:1215.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00029014796018600464 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8514383714646101) - timing_s/agent_loop/generate_sequences/max:np.float64(10.650067988783121) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.062247704801848) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.650067988783121) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1204 - timing_s/gen:17.77479348704219 - timing_s/reward:0.06109056994318962 - timing_s/old_log_prob:3.223510628566146 - timing_s/adv:0.6315231639891863 - timing_s/update_actor:22.08882056735456 - timing_s/step:43.915783969685435 - timing_s/stop_profile:4.657916724681854e-05 - timing_per_token_ms/gen:0.13923105563856833 - timing_per_token_ms/update_actor:0.11101248676903024 - timing_per_token_ms/adv:0.0031738660139372906 - perf/total_num_tokens:198976 - perf/time_per_step:43.915783969685435 - perf/throughput:566.356734452671 (TaskRunner pid=2320396) Training Progress: 45%|████▌ | 45/100 [43:56<47:45, 52.10s/it] (TaskRunner pid=2320396) step:46 - global_seqlen/min:15575 - global_seqlen/max:30167 - global_seqlen/minmax_diff:14592 - global_seqlen/balanced_min:22514 - global_seqlen/balanced_max:22538 - global_seqlen/mean:22530.625 - actor/entropy:0.21788272261619568 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967324733734131 - training/rollout_probs_diff_mean:0.002842165995389223 - training/rollout_probs_diff_std:0.009226741269230843 - training/rollout_actor_probs_pearson_corr:0.9992075562477112 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999492168426514 - rollout_corr/rollout_is_ratio_fraction_high:2.5889521566568874e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.314920079195872e-05 - rollout_corr/rollout_is_max:2.2939116954803467 - rollout_corr/rollout_is_min:0.26859140396118164 - rollout_corr/rollout_is_std:0.02792610228061676 - rollout_corr/rollout_is_eff_sample_size:0.9992207405466251 - rollout_corr/rollout_is_seq_mean:1.0000059604644775 - rollout_corr/rollout_is_seq_std:0.0019772870000451803 - rollout_corr/rollout_is_seq_max:1.0133771896362305 - rollout_corr/rollout_is_seq_min:0.9926490783691406 - rollout_corr/rollout_is_seq_max_deviation:0.013377189636230469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.231029165443033) - rollout_corr/training_log_ppl:np.float64(0.19566736919659888) - rollout_corr/kl:np.float64(0.0004418784923347907) - rollout_corr/k3_kl:np.float64(0.0004674396851669371) - rollout_corr/rollout_ppl:np.float64(1.230477964039892) - rollout_corr/rollout_log_ppl:np.float64(0.19522549020621227) - rollout_corr/log_ppl_diff:np.float64(0.0004418789903866127) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013463952200254425) - rollout_corr/log_ppl_diff_max:np.float64(0.009284377098083496) - rollout_corr/log_ppl_diff_min:np.float64(-0.01020503044128418) - rollout_corr/ppl_ratio:np.float64(1.000443896977231) - rollout_corr/chi2_token:np.float64(0.0009975701104849577) - rollout_corr/chi2_seq:np.float64(0.27493038563989103) - actor/pg_loss:np.float64(0.0021532203536480665) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017491115090706444) - actor/ppo_kl:np.float64(3.2276031305400465e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(4078.0635640691034) - self_distillation/token_reweight_w_std:np.float64(119085.47364705714) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001369349192828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.044050666445400566) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.1551105249673128) - perf/mfu/actor:np.float64(0.053146217542449595) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.63757705688477) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01592421345412731 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01592421345412731 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:452.64453125 - response_length/max:1570.0 - response_length/min:60.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:452.64453125 - response_length_non_aborted/max:1570.0 - response_length_non_aborted/min:60.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2947551012039185e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7519746031612158) - timing_s/agent_loop/generate_sequences/max:np.float64(12.470940450206399) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.772140024790133) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.470940450206399) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1570 - timing_s/gen:19.47098889015615 - timing_s/reward:0.05715354159474373 - timing_s/old_log_prob:2.9865529257804155 - timing_s/adv:0.6135058924555779 - timing_s/update_actor:21.954095283523202 - timing_s/step:45.127909518778324 - timing_s/stop_profile:2.9237940907478333e-05 - timing_per_token_ms/gen:0.16803152385854095 - timing_per_token_ms/update_actor:0.1218014107660307 - timing_per_token_ms/adv:0.0034037332101061212 - perf/total_num_tokens:180245 - perf/time_per_step:45.127909518778324 - perf/throughput:499.2614379938142 (TaskRunner pid=2320396) Training Progress: 46%|████▌ | 46/100 [44:41<45:00, 50.01s/it] (TaskRunner pid=2320396) step:47 - global_seqlen/min:21565 - global_seqlen/max:31125 - global_seqlen/minmax_diff:9560 - global_seqlen/balanced_min:25397 - global_seqlen/balanced_max:25437 - global_seqlen/mean:25423.0 - actor/entropy:0.20809735357761383 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37773603200912476 - training/rollout_probs_diff_mean:0.002722726669162512 - training/rollout_probs_diff_std:0.008913216181099415 - training/rollout_actor_probs_pearson_corr:0.999236524105072 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9998671412467957 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9537394046783447 - rollout_corr/rollout_is_min:0.5497885942459106 - rollout_corr/rollout_is_std:0.02662140503525734 - rollout_corr/rollout_is_eff_sample_size:0.9992915645915715 - rollout_corr/rollout_is_seq_mean:0.9997721314430237 - rollout_corr/rollout_is_seq_std:0.0015414318768307567 - rollout_corr/rollout_is_seq_max:1.005649447441101 - rollout_corr/rollout_is_seq_min:0.9940074682235718 - rollout_corr/rollout_is_seq_max_deviation:0.005992531776428223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2369549586437643) - rollout_corr/training_log_ppl:np.float64(0.20200137386564165) - rollout_corr/kl:np.float64(0.0005988720004523884) - rollout_corr/k3_kl:np.float64(0.0004104271441178753) - rollout_corr/rollout_ppl:np.float64(1.2361970636993647) - rollout_corr/rollout_log_ppl:np.float64(0.201402501334087) - rollout_corr/log_ppl_diff:np.float64(0.0005988725315546617) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012694383185589686) - rollout_corr/log_ppl_diff_max:np.float64(0.0073181986808776855) - rollout_corr/log_ppl_diff_min:np.float64(-0.00609000027179718) - rollout_corr/ppl_ratio:np.float64(1.0006005712784827) - rollout_corr/chi2_token:np.float64(0.0004317518323659897) - rollout_corr/chi2_seq:np.float64(0.32166874944232404) - actor/pg_loss:np.float64(0.0022747202310711145) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001154373726421909) - actor/ppo_kl:np.float64(-2.0942095368292613e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(300.36585143371485) - self_distillation/token_reweight_w_std:np.float64(6000.846875203308) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002250391989946) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03434523963369429) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.13008191250264645) - perf/mfu/actor:np.float64(0.05954238275847209) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.58963775634766) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004297312814742327 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004297312814742327 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:529.71875 - response_length/max:1681.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:529.71875 - response_length_non_aborted/max:1681.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.859058678150177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8061241153627634) - timing_s/agent_loop/generate_sequences/max:np.float64(13.01833027601242) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.616919153006165) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.01833027601242) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1681 - timing_s/gen:20.68991795182228 - timing_s/reward:0.06652592867612839 - timing_s/old_log_prob:3.264441102743149 - timing_s/adv:0.6679162699729204 - timing_s/update_actor:22.31015120446682 - timing_s/step:47.09043016284704 - timing_s/stop_profile:0.0001293029636144638 - timing_per_token_ms/gen:0.15257151459959797 - timing_per_token_ms/update_actor:0.10969472133730687 - timing_per_token_ms/adv:0.00328401580248653 - perf/total_num_tokens:203384 - perf/time_per_step:47.09043016284704 - perf/throughput:539.8761470660337 (TaskRunner pid=2320396) Training Progress: 47%|████▋ | 47/100 [45:28<43:25, 49.16s/it] (TaskRunner pid=2320396) step:48 - global_seqlen/min:15612 - global_seqlen/max:27953 - global_seqlen/minmax_diff:12341 - global_seqlen/balanced_min:23320 - global_seqlen/balanced_max:23770 - global_seqlen/mean:23382.0 - actor/entropy:0.20437420904636383 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2153259515762329 - training/rollout_probs_diff_mean:0.002861631801351905 - training/rollout_probs_diff_std:0.009165971539914608 - training/rollout_actor_probs_pearson_corr:0.9991044402122498 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9999480843544006 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.373819345142692e-05 - rollout_corr/rollout_is_max:1.8078258037567139 - rollout_corr/rollout_is_min:0.1487177610397339 - rollout_corr/rollout_is_std:0.026906467974185944 - rollout_corr/rollout_is_eff_sample_size:0.9992764466837457 - rollout_corr/rollout_is_seq_mean:0.9999303221702576 - rollout_corr/rollout_is_seq_std:0.0016568535938858986 - rollout_corr/rollout_is_seq_max:1.0050488710403442 - rollout_corr/rollout_is_seq_min:0.9914178252220154 - rollout_corr/rollout_is_seq_max_deviation:0.00858217477798462 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2299613454379141) - rollout_corr/training_log_ppl:np.float64(0.19597684718610253) - rollout_corr/kl:np.float64(0.0004885526310606236) - rollout_corr/k3_kl:np.float64(0.0004654208404986093) - rollout_corr/rollout_ppl:np.float64(1.2293341453187168) - rollout_corr/rollout_log_ppl:np.float64(0.19548829441191629) - rollout_corr/log_ppl_diff:np.float64(0.0004885527741862461) - rollout_corr/log_ppl_abs_diff:np.float64(0.001387346550473012) - rollout_corr/log_ppl_diff_max:np.float64(0.006539776921272278) - rollout_corr/log_ppl_diff_min:np.float64(-0.006212443113327026) - rollout_corr/ppl_ratio:np.float64(1.0004903129301965) - rollout_corr/chi2_token:np.float64(0.0008849361911416054) - rollout_corr/chi2_seq:np.float64(0.5547701527830213) - actor/pg_loss:np.float64(0.0020671726670116186) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001482926384142047) - actor/ppo_kl:np.float64(7.87036429628074e-06) - actor/pg_clipfrac_lower:np.float64(7.038288458716124e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(3750.297277679201) - self_distillation/token_reweight_w_std:np.float64(98113.6687944993) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994098702445626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04493811877910048) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.17269694805145264) - perf/mfu/actor:np.float64(0.05559806881598567) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.6170768737793) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016268134117126465 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016268134117126465 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:463.125 - response_length/max:2002.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.125 - response_length_non_aborted/max:2002.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.217990398406982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2065272722393274) - timing_s/agent_loop/generate_sequences/max:np.float64(14.751506268978119) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.25816673142981) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.751506268978119) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:2002 - timing_s/gen:21.466947183012962 - timing_s/reward:0.06394496373832226 - timing_s/old_log_prob:3.1414225921034813 - timing_s/adv:0.5687259417027235 - timing_s/update_actor:21.953529139980674 - timing_s/step:47.28480998799205 - timing_s/stop_profile:0.00011818297207355499 - timing_per_token_ms/gen:0.18106399445861135 - timing_per_token_ms/update_actor:0.117363405290291 - timing_per_token_ms/adv:0.0030404047007458915 - perf/total_num_tokens:187056 - perf/time_per_step:47.28480998799205 - perf/throughput:494.4928404267215 (TaskRunner pid=2320396) Training Progress: 48%|████▊ | 48/100 [46:16<42:07, 48.61s/it] (TaskRunner pid=2320396) step:49 - global_seqlen/min:16311 - global_seqlen/max:29943 - global_seqlen/minmax_diff:13632 - global_seqlen/balanced_min:23478 - global_seqlen/balanced_max:23546 - global_seqlen/mean:23497.5 - actor/entropy:0.2099330872297287 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583489418029785 - training/rollout_probs_diff_mean:0.002830114448443055 - training/rollout_probs_diff_std:0.009117886424064636 - training/rollout_actor_probs_pearson_corr:0.9990902543067932 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6360359191894531 - rollout_corr/rollout_is_min:0.5087555050849915 - rollout_corr/rollout_is_std:0.027530577033758163 - rollout_corr/rollout_is_eff_sample_size:0.9992427603926878 - rollout_corr/rollout_is_seq_mean:1.0000454187393188 - rollout_corr/rollout_is_seq_std:0.001672466634772718 - rollout_corr/rollout_is_seq_max:1.0071189403533936 - rollout_corr/rollout_is_seq_min:0.9933210611343384 - rollout_corr/rollout_is_seq_max_deviation:0.007118940353393555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2388078104704618) - rollout_corr/training_log_ppl:np.float64(0.19887281095725484) - rollout_corr/kl:np.float64(0.00033442378822035046) - rollout_corr/k3_kl:np.float64(0.0004594542516542788) - rollout_corr/rollout_ppl:np.float64(1.2383832414634526) - rollout_corr/rollout_log_ppl:np.float64(0.19853838607377838) - rollout_corr/log_ppl_diff:np.float64(0.0003344248834764585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013376152055570856) - rollout_corr/log_ppl_diff_max:np.float64(0.006600901484489441) - rollout_corr/log_ppl_diff_min:np.float64(-0.006887331604957581) - rollout_corr/ppl_ratio:np.float64(1.000336108962074) - rollout_corr/chi2_token:np.float64(0.0011650859378278255) - rollout_corr/chi2_seq:np.float64(0.6430712481960654) - actor/pg_loss:np.float64(0.0028300664853304625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023176581339612312) - actor/ppo_kl:np.float64(-4.170142055848203e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_w_mean:np.float64(5994.0513384339865) - self_distillation/token_reweight_w_std:np.float64(110882.28832101094) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000301810214296) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04926858190447092) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.16475248709321022) - perf/mfu/actor:np.float64(0.05549702752766853) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.6996307373047) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001991107128560543 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001991107128560543 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:479.671875 - response_length/max:1847.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:479.671875 - response_length_non_aborted/max:1847.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010908395051956177 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7865940872579813) - timing_s/agent_loop/generate_sequences/max:np.float64(13.637167720124125) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.768220525184006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.637167720124125) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1815 - timing_s/gen:20.72220243141055 - timing_s/reward:0.06068837456405163 - timing_s/old_log_prob:3.109410423785448 - timing_s/adv:0.724044319242239 - timing_s/update_actor:22.068346058949828 - timing_s/step:46.765394140034914 - timing_s/stop_profile:0.00011418946087360382 - timing_per_token_ms/gen:0.16875307364580727 - timing_per_token_ms/update_actor:0.11739730853787546 - timing_per_token_ms/adv:0.0038517093267487978 - perf/total_num_tokens:187980 - perf/time_per_step:46.765394140034914 - perf/throughput:502.4548692915701 (TaskRunner pid=2320396) Training Progress: 49%|████▉ | 49/100 [47:03<40:51, 48.07s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) We solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2320396) step:50 - global_seqlen/min:19168 - global_seqlen/max:29440 - global_seqlen/minmax_diff:10272 - global_seqlen/balanced_min:25436 - global_seqlen/balanced_max:25450 - global_seqlen/mean:25439.375 - actor/entropy:0.27149584889411926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37974900007247925 - training/rollout_probs_diff_mean:0.0030482823494821787 - training/rollout_probs_diff_std:0.009097245521843433 - training/rollout_actor_probs_pearson_corr:0.999318540096283 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999427199363708 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4786446627113037e-05 - rollout_corr/rollout_is_max:1.8260700702667236 - rollout_corr/rollout_is_min:0.42270684242248535 - rollout_corr/rollout_is_std:0.02846306934952736 - rollout_corr/rollout_is_eff_sample_size:0.9991903904576372 - rollout_corr/rollout_is_seq_mean:0.9999755024909973 - rollout_corr/rollout_is_seq_std:0.0014398901257663965 - rollout_corr/rollout_is_seq_max:1.0053409337997437 - rollout_corr/rollout_is_seq_min:0.994831383228302 - rollout_corr/rollout_is_seq_max_deviation:0.005340933799743652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3109961580485106) - rollout_corr/training_log_ppl:np.float64(0.253154604928568) - rollout_corr/kl:np.float64(0.00036529192135503763) - rollout_corr/k3_kl:np.float64(0.0004534678223535593) - rollout_corr/rollout_ppl:np.float64(1.3104890743270516) - rollout_corr/rollout_log_ppl:np.float64(0.2527893086516997) - rollout_corr/log_ppl_diff:np.float64(0.000365296276868321) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012172957794973627) - rollout_corr/log_ppl_diff_max:np.float64(0.006820358335971832) - rollout_corr/log_ppl_diff_min:np.float64(-0.006113961338996887) - rollout_corr/ppl_ratio:np.float64(1.0003666609991342) - rollout_corr/chi2_token:np.float64(0.0010919254273176193) - rollout_corr/chi2_seq:np.float64(0.44652247754856944) - actor/pg_loss:np.float64(0.004591187578625977) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002630960186706943) - actor/ppo_kl:np.float64(-8.892219416978264e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(3386.0691014186013) - self_distillation/token_reweight_w_std:np.float64(81954.25167127652) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989137353841215) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09482813796785194) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2807828411459923) - perf/mfu/actor:np.float64(0.05904541053648982) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.66108226776123) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.20277213467419983) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7293499999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17546086487058532) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5562374999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16899377189330053) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.641375) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20366229536255137) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8003374999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12873103397768565) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4905375) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1195036280366466) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.662125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1700630423843838) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8505125) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0808261666845073) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.445125) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07380946263817979) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6723125000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12969198776088287) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8791499999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04230892033532858) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4182125) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.040390744834128765) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6767) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09448682040636207) - val-aux/sciknoweval/score/mean@16:np.float64(0.6421875) - val-aux/sciknoweval/score/std@16:np.float64(0.20277213467419983) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7293499999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17546086487058532) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5562374999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16899377189330053) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.641375) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20366229536255137) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8003374999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12873103397768565) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4905375) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1195036280366466) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.662125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1700630423843838) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8505125) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0808261666845073) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.445125) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07380946263817979) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6723125000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12969198776088287) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8791499999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04230892033532858) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4182125) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.040390744834128765) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6767) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09448682040636207) - val-core/sciknoweval/acc/mean@16:np.float64(0.6421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.20277213467419983) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7293499999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17546086487058532) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5562374999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16899377189330053) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.641375) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20366229536255137) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8003374999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12873103397768565) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4905375) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1195036280366466) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.662125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1700630423843838) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8505125) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0808261666845073) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.445125) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07380946263817979) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6723125000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12969198776088287) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8791499999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04230892033532858) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4182125) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.040390744834128765) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6767) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09448682040636207) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98671875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03262356191773175) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9967750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013393238265410088) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.977025) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04044627703253543) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.986825) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.032195435957587636) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9994125) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0030136505532463915) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.96235) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.046578683223695114) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9938750000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021421915469836496) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9415125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.049659373319305505) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9979375000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011968339769143646) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9173) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04406214749247993) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998000000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0015684387141358125) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.019525503739714622 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.019525503739714622 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:528.35546875 - response_length/max:1825.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:528.35546875 - response_length_non_aborted/max:1825.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001132078468799591 - timing_s/agent_loop/generate_sequences/min:np.float64(0.971835458651185) - timing_s/agent_loop/generate_sequences/max:np.float64(14.26541281864047) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.613275811665517) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.26541281864047) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1825 - timing_s/gen:20.7291506472975 - timing_s/reward:0.06667716056108475 - timing_s/old_log_prob:3.100318057462573 - timing_s/adv:0.6697617433965206 - timing_s/update_actor:22.497836330905557 - timing_s/step:47.153244730085135 - timing_s/testing:112.05036744847894 - timing_s/save_checkpoint:12.590073201805353 - timing_s/stop_profile:0.00015703029930591583 - timing_per_token_ms/gen:0.15325524103606786 - timing_per_token_ms/update_actor:0.11054632990642241 - timing_per_token_ms/adv:0.0032909699206275737 - perf/total_num_tokens:203515 - perf/time_per_step:47.153244730085135 - perf/throughput:539.5042302098235 (TaskRunner pid=2320396) Training Progress: 50%|█████ | 50/100 [49:54<1:11:00, 85.21s/it] (TaskRunner pid=2320396) step:51 - global_seqlen/min:17511 - global_seqlen/max:28028 - global_seqlen/minmax_diff:10517 - global_seqlen/balanced_min:23155 - global_seqlen/balanced_max:23188 - global_seqlen/mean:23179.125 - actor/entropy:0.2698791027069092 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38078969717025757 - training/rollout_probs_diff_mean:0.0030010524205863476 - training/rollout_probs_diff_std:0.00926926452666521 - training/rollout_actor_probs_pearson_corr:0.99923175573349 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001506805419922 - rollout_corr/rollout_is_ratio_fraction_high:8.544598131265957e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.544598131265957e-06 - rollout_corr/rollout_is_max:2.1633248329162598 - rollout_corr/rollout_is_min:0.3055838942527771 - rollout_corr/rollout_is_std:0.028341351076960564 - rollout_corr/rollout_is_eff_sample_size:0.999197769525319 - rollout_corr/rollout_is_seq_mean:1.0000841617584229 - rollout_corr/rollout_is_seq_std:0.0014276247238740325 - rollout_corr/rollout_is_seq_max:1.0039843320846558 - rollout_corr/rollout_is_seq_min:0.9958688020706177 - rollout_corr/rollout_is_seq_max_deviation:0.004131197929382324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.283448894508183) - rollout_corr/training_log_ppl:np.float64(0.23242870725516696) - rollout_corr/kl:np.float64(0.00039944934866742443) - rollout_corr/k3_kl:np.float64(0.0004781505269306763) - rollout_corr/rollout_ppl:np.float64(1.2829442485235631) - rollout_corr/rollout_log_ppl:np.float64(0.23202925745863467) - rollout_corr/log_ppl_diff:np.float64(0.00039944979653228074) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011975742381764576) - rollout_corr/log_ppl_diff_max:np.float64(0.005612224340438843) - rollout_corr/log_ppl_diff_min:np.float64(-0.0040902793407440186) - rollout_corr/ppl_ratio:np.float64(1.000400707591325) - rollout_corr/chi2_token:np.float64(0.0011282062623649836) - rollout_corr/chi2_seq:np.float64(0.39218471315689385) - actor/pg_loss:np.float64(0.002458873437717557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036319168407317193) - actor/ppo_kl:np.float64(6.448495334332449e-05) - actor/pg_clipfrac_lower:np.float64(7.689468475291505e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(5380.610399987083) - self_distillation/token_reweight_w_std:np.float64(106550.12923576194) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000015466939658) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06583685430814512) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.19366929307579994) - perf/mfu/actor:np.float64(0.05463014221907142) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.95783233642578) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001417335239239037 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001417335239239037 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.16015625 - response_length/max:1647.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.16015625 - response_length_non_aborted/max:1647.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.342469274997711e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1378375869244337) - timing_s/agent_loop/generate_sequences/max:np.float64(12.7565373852849) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.867362762641278) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.7565373852849) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1647 - timing_s/gen:19.348504519090056 - timing_s/reward:0.06200408935546875 - timing_s/old_log_prob:3.208234280347824 - timing_s/adv:0.5986099690198898 - timing_s/update_actor:22.022641595453024 - timing_s/step:45.33466789312661 - timing_s/stop_profile:9.629316627979279e-05 - timing_per_token_ms/gen:0.16532520331094697 - timing_per_token_ms/update_actor:0.11876333551985366 - timing_per_token_ms/adv:0.0032281738904072622 - perf/total_num_tokens:185433 - perf/time_per_step:45.33466789312661 - perf/throughput:511.2891761916776 (TaskRunner pid=2320396) Training Progress: 51%|█████ | 51/100 [50:40<59:49, 73.26s/it] (TaskRunner pid=2320396) step:52 - global_seqlen/min:20249 - global_seqlen/max:26904 - global_seqlen/minmax_diff:6655 - global_seqlen/balanced_min:23741 - global_seqlen/balanced_max:23754 - global_seqlen/mean:23748.875 - actor/entropy:0.26206788420677185 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2897759675979614 - training/rollout_probs_diff_mean:0.0031796968542039394 - training/rollout_probs_diff_std:0.009452592581510544 - training/rollout_actor_probs_pearson_corr:0.9992661476135254 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9998950362205505 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.323414476762991e-06 - rollout_corr/rollout_is_max:1.6328537464141846 - rollout_corr/rollout_is_min:0.2245272547006607 - rollout_corr/rollout_is_std:0.02913775108754635 - rollout_corr/rollout_is_eff_sample_size:0.9991514736296966 - rollout_corr/rollout_is_seq_mean:0.9998642206192017 - rollout_corr/rollout_is_seq_std:0.0018541476456448436 - rollout_corr/rollout_is_seq_max:1.0078346729278564 - rollout_corr/rollout_is_seq_min:0.990766167640686 - rollout_corr/rollout_is_seq_max_deviation:0.009233832359313965 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2924986188299954) - rollout_corr/training_log_ppl:np.float64(0.24121209807344712) - rollout_corr/kl:np.float64(0.0005627371857137575) - rollout_corr/k3_kl:np.float64(0.000526310548963238) - rollout_corr/rollout_ppl:np.float64(1.2917516590096056) - rollout_corr/rollout_log_ppl:np.float64(0.24064936087233946) - rollout_corr/log_ppl_diff:np.float64(0.0005627372011076659) - rollout_corr/log_ppl_abs_diff:np.float64(0.001493871764978394) - rollout_corr/log_ppl_diff_max:np.float64(0.008751854300498962) - rollout_corr/log_ppl_diff_min:np.float64(-0.008186154067516327) - rollout_corr/ppl_ratio:np.float64(1.000564940739423) - rollout_corr/chi2_token:np.float64(0.0010216198861598969) - rollout_corr/chi2_seq:np.float64(0.5899208602495492) - actor/pg_loss:np.float64(0.0034945248626172543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027937310051129316) - actor/ppo_kl:np.float64(-5.135034418657369e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(15624.9816659051) - self_distillation/token_reweight_w_std:np.float64(329907.70011061645) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006199525669217) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06881534481362905) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2124406099319458) - perf/mfu/actor:np.float64(0.05564996634226216) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.88951110839844) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004192920168861747 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004192920168861747 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:469.30859375 - response_length/max:1387.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:469.30859375 - response_length_non_aborted/max:1387.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.375089287757874e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0267762243747711) - timing_s/agent_loop/generate_sequences/max:np.float64(11.22788168117404) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.981896769080777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.22788168117404) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1387 - timing_s/gen:18.050167409703135 - timing_s/reward:0.06669834069907665 - timing_s/old_log_prob:2.978366069495678 - timing_s/adv:0.6107301041483879 - timing_s/update_actor:21.93880164436996 - timing_s/step:43.7340450193733 - timing_s/stop_profile:0.00011033564805984497 - timing_per_token_ms/gen:0.15023902690712848 - timing_per_token_ms/update_actor:0.11547284684206073 - timing_per_token_ms/adv:0.003214521235997431 - perf/total_num_tokens:189991 - perf/time_per_step:43.7340450193733 - perf/throughput:543.0294634187103 (TaskRunner pid=2320396) Training Progress: 52%|█████▏ | 52/100 [51:24<51:31, 64.41s/it] (TaskRunner pid=2320396) step:53 - global_seqlen/min:17256 - global_seqlen/max:25754 - global_seqlen/minmax_diff:8498 - global_seqlen/balanced_min:22977 - global_seqlen/balanced_max:22994 - global_seqlen/mean:22989.0 - actor/entropy:0.230458065867424 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44532376527786255 - training/rollout_probs_diff_mean:0.0028626120183616877 - training/rollout_probs_diff_std:0.009131461381912231 - training/rollout_actor_probs_pearson_corr:0.999158501625061 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000241994857788 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.838919711706694e-06 - rollout_corr/rollout_is_max:1.7926321029663086 - rollout_corr/rollout_is_min:0.4609784185886383 - rollout_corr/rollout_is_std:0.02715124748647213 - rollout_corr/rollout_is_eff_sample_size:0.9992634718362852 - rollout_corr/rollout_is_seq_mean:1.000015377998352 - rollout_corr/rollout_is_seq_std:0.0015237050829455256 - rollout_corr/rollout_is_seq_max:1.0050256252288818 - rollout_corr/rollout_is_seq_min:0.9932184815406799 - rollout_corr/rollout_is_seq_max_deviation:0.006781518459320068 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2424569255672395) - rollout_corr/training_log_ppl:np.float64(0.20219183068547864) - rollout_corr/kl:np.float64(0.00047282653207858516) - rollout_corr/k3_kl:np.float64(0.00042950060326774064) - rollout_corr/rollout_ppl:np.float64(1.241812152788043) - rollout_corr/rollout_log_ppl:np.float64(0.20171900314744562) - rollout_corr/log_ppl_diff:np.float64(0.000472827538033016) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012220797216286883) - rollout_corr/log_ppl_diff_max:np.float64(0.01025158166885376) - rollout_corr/log_ppl_diff_min:np.float64(-0.00387602299451828) - rollout_corr/ppl_ratio:np.float64(1.000474382424727) - rollout_corr/chi2_token:np.float64(0.0006975317373871803) - rollout_corr/chi2_seq:np.float64(0.14690965181216598) - actor/pg_loss:np.float64(0.0025837309658527374) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020541453568512225) - actor/ppo_kl:np.float64(0.00013539861986799906) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(1052.9588372900616) - self_distillation/token_reweight_w_std:np.float64(28003.405932930298) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998704448807985) - self_distillation/token_reweight_w_clip_frac:np.float64(0.055031316907843575) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.15601524617522955) - perf/mfu/actor:np.float64(0.054306824026305565) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7872657775879) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014584216987714171 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014584216987714171 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:441.9375 - response_length/max:1407.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:441.9375 - response_length_non_aborted/max:1407.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001513790339231491 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0363005455583334) - timing_s/agent_loop/generate_sequences/max:np.float64(11.363907653838396) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.789611810541828) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.363907653838396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:256 - timing_s/agent_loop/slowest/response_length:1407 - timing_s/gen:18.20484925620258 - timing_s/reward:0.0619586817920208 - timing_s/old_log_prob:3.02254974283278 - timing_s/adv:0.648742325603962 - timing_s/update_actor:21.905866038054228 - timing_s/step:43.93234056420624 - timing_s/stop_profile:0.00011724047362804413 - timing_per_token_ms/gen:0.16091119764003128 - timing_per_token_ms/update_actor:0.11911058570432723 - timing_per_token_ms/adv:0.0035274605550696093 - perf/total_num_tokens:183912 - perf/time_per_step:43.93234056420624 - perf/throughput:523.2819309137885 (TaskRunner pid=2320396) Training Progress: 53%|█████▎ | 53/100 [52:08<45:39, 58.29s/it] (TaskRunner pid=2320396) step:54 - global_seqlen/min:17834 - global_seqlen/max:25657 - global_seqlen/minmax_diff:7823 - global_seqlen/balanced_min:21174 - global_seqlen/balanced_max:21207 - global_seqlen/mean:21197.75 - actor/entropy:0.23790472745895386 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3006957769393921 - training/rollout_probs_diff_mean:0.0029614968225359917 - training/rollout_probs_diff_std:0.009239520877599716 - training/rollout_actor_probs_pearson_corr:0.9991896152496338 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000056028366089 - rollout_corr/rollout_is_ratio_fraction_high:9.66090192378033e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.932180384756066e-05 - rollout_corr/rollout_is_max:2.61974835395813 - rollout_corr/rollout_is_min:0.3203427791595459 - rollout_corr/rollout_is_std:0.02868834137916565 - rollout_corr/rollout_is_eff_sample_size:0.9991775368580812 - rollout_corr/rollout_is_seq_mean:0.9999408721923828 - rollout_corr/rollout_is_seq_std:0.0016452306881546974 - rollout_corr/rollout_is_seq_max:1.0042672157287598 - rollout_corr/rollout_is_seq_min:0.9935493469238281 - rollout_corr/rollout_is_seq_max_deviation:0.006450653076171875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.244489336386323) - rollout_corr/training_log_ppl:np.float64(0.20888344780541956) - rollout_corr/kl:np.float64(0.0004943577070619654) - rollout_corr/k3_kl:np.float64(0.00044278011073117796) - rollout_corr/rollout_ppl:np.float64(1.243890237994492) - rollout_corr/rollout_log_ppl:np.float64(0.20838908667792566) - rollout_corr/log_ppl_diff:np.float64(0.000494361127493903) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013888106041122228) - rollout_corr/log_ppl_diff_max:np.float64(0.008533328771591187) - rollout_corr/log_ppl_diff_min:np.float64(-0.005737587809562683) - rollout_corr/ppl_ratio:np.float64(1.0004961658269167) - rollout_corr/chi2_token:np.float64(0.0007693876978009939) - rollout_corr/chi2_seq:np.float64(0.3788678168784827) - actor/pg_loss:np.float64(0.0019641465041786432) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015006033072495484) - actor/ppo_kl:np.float64(-2.1946025228203325e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(1024.2289772662334) - self_distillation/token_reweight_w_std:np.float64(25291.225553450407) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008950042538345) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04413453795132227) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.13687882665544748) - perf/mfu/actor:np.float64(0.050549113817286606) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.81171035766602) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004241135902702808 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.004241135902702808 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:404.3359375 - response_length/max:1449.0 - response_length/min:65.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.3359375 - response_length_non_aborted/max:1449.0 - response_length_non_aborted/min:65.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.37432050704956e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7421483434736729) - timing_s/agent_loop/generate_sequences/max:np.float64(10.745056714862585) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.232009753388411) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.745056714862585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1449 - timing_s/gen:18.027256151661277 - timing_s/reward:0.07766947150230408 - timing_s/old_log_prob:3.0949815083295107 - timing_s/adv:0.6324186753481627 - timing_s/update_actor:21.854929784312844 - timing_s/step:43.77818885631859 - timing_s/stop_profile:8.252449333667755e-05 - timing_per_token_ms/gen:0.1741595609280386 - timing_per_token_ms/update_actor:0.12887529209652465 - timing_per_token_ms/adv:0.0037292794951596436 - perf/total_num_tokens:169582 - perf/time_per_step:43.77818885631859 - perf/throughput:484.2080166809022 (TaskRunner pid=2320396) Training Progress: 54%|█████▍ | 54/100 [52:51<41:21, 53.95s/it] (TaskRunner pid=2320396) step:55 - global_seqlen/min:15068 - global_seqlen/max:26886 - global_seqlen/minmax_diff:11818 - global_seqlen/balanced_min:21975 - global_seqlen/balanced_max:21993 - global_seqlen/mean:21988.75 - actor/entropy:0.17557011544704437 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29323479533195496 - training/rollout_probs_diff_mean:0.0027691253926604986 - training/rollout_probs_diff_std:0.00955580361187458 - training/rollout_actor_probs_pearson_corr:0.9989780187606812 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:1.000060796737671 - rollout_corr/rollout_is_ratio_fraction_high:9.256002158508636e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.7768008294515312e-05 - rollout_corr/rollout_is_max:2.0412697792053223 - rollout_corr/rollout_is_min:0.40601131319999695 - rollout_corr/rollout_is_std:0.02833504043519497 - rollout_corr/rollout_is_eff_sample_size:0.9991978885434327 - rollout_corr/rollout_is_seq_mean:1.0000414848327637 - rollout_corr/rollout_is_seq_std:0.00220958748832345 - rollout_corr/rollout_is_seq_max:1.0099327564239502 - rollout_corr/rollout_is_seq_min:0.9892379641532898 - rollout_corr/rollout_is_seq_max_deviation:0.010762035846710205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2014198061078787) - rollout_corr/training_log_ppl:np.float64(0.17472496219124878) - rollout_corr/kl:np.float64(0.0003758088305049201) - rollout_corr/k3_kl:np.float64(0.0004877579008848443) - rollout_corr/rollout_ppl:np.float64(1.2009721314534545) - rollout_corr/rollout_log_ppl:np.float64(0.17434915404010098) - rollout_corr/log_ppl_diff:np.float64(0.000375808151147794) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015464641401194967) - rollout_corr/log_ppl_diff_max:np.float64(0.014139816164970398) - rollout_corr/log_ppl_diff_min:np.float64(-0.009086787700653076) - rollout_corr/ppl_ratio:np.float64(1.0003787553869188) - rollout_corr/chi2_token:np.float64(0.0011782660149037838) - rollout_corr/chi2_seq:np.float64(0.4314419652801007) - actor/pg_loss:np.float64(0.0012743137776851654) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001162844778264116) - actor/ppo_kl:np.float64(-5.910023932820252e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(9333.47411274095) - self_distillation/token_reweight_w_std:np.float64(244293.20970271114) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000400455435738) - self_distillation/token_reweight_w_clip_frac:np.float64(0.033853263084893115) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.10476882196962833) - perf/mfu/actor:np.float64(0.052601070214590546) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.73566818237305) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00026263907784596086 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00026263907784596086 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:422.0234375 - response_length/max:1367.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:422.0234375 - response_length_non_aborted/max:1367.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.639561176300049e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0234655812382698) - timing_s/agent_loop/generate_sequences/max:np.float64(10.340585360303521) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.485789840742655) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.340585360303521) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1367 - timing_s/gen:17.07848149165511 - timing_s/reward:0.06134553626179695 - timing_s/old_log_prob:3.1349700447171926 - timing_s/adv:0.5768703576177359 - timing_s/update_actor:21.743792919442058 - timing_s/step:42.68766647577286 - timing_s/stop_profile:0.00012461841106414795 - timing_per_token_ms/gen:0.1580784676841029 - timing_per_token_ms/update_actor:0.12360748632506428 - timing_per_token_ms/adv:0.0032793494265120566 - perf/total_num_tokens:175910 - perf/time_per_step:42.68766647577286 - perf/throughput:515.1078008089196 (TaskRunner pid=2320396) Training Progress: 55%|█████▌ | 55/100 [53:34<37:56, 50.59s/it] (TaskRunner pid=2320396) step:56 - global_seqlen/min:18311 - global_seqlen/max:32456 - global_seqlen/minmax_diff:14145 - global_seqlen/balanced_min:22088 - global_seqlen/balanced_max:22120 - global_seqlen/mean:22110.5 - actor/entropy:0.21576471626758575 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19307291507720947 - training/rollout_probs_diff_mean:0.0028168810531497 - training/rollout_probs_diff_std:0.009117874316871166 - training/rollout_actor_probs_pearson_corr:0.9991778135299683 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.921875 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.921875 - rollout_corr/rollout_is_mean:1.0000052452087402 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9838331937789917 - rollout_corr/rollout_is_min:0.5898699164390564 - rollout_corr/rollout_is_std:0.027591131627559662 - rollout_corr/rollout_is_eff_sample_size:0.9992394275969021 - rollout_corr/rollout_is_seq_mean:1.0000158548355103 - rollout_corr/rollout_is_seq_std:0.0018200587946921587 - rollout_corr/rollout_is_seq_max:1.0058069229125977 - rollout_corr/rollout_is_seq_min:0.9924702644348145 - rollout_corr/rollout_is_seq_max_deviation:0.007529735565185547 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2427338524721563) - rollout_corr/training_log_ppl:np.float64(0.20696487333043478) - rollout_corr/kl:np.float64(0.0005580308752195229) - rollout_corr/k3_kl:np.float64(0.0004463659485622884) - rollout_corr/rollout_ppl:np.float64(1.2419886952266097) - rollout_corr/rollout_log_ppl:np.float64(0.20640684351383243) - rollout_corr/log_ppl_diff:np.float64(0.0005580298166023567) - rollout_corr/log_ppl_abs_diff:np.float64(0.00135869525547605) - rollout_corr/log_ppl_diff_max:np.float64(0.0075851380825042725) - rollout_corr/log_ppl_diff_min:np.float64(-0.0047438740730285645) - rollout_corr/ppl_ratio:np.float64(1.0005598990246654) - rollout_corr/chi2_token:np.float64(0.0006612758152186871) - rollout_corr/chi2_seq:np.float64(0.2137361967470497) - actor/pg_loss:np.float64(0.00270246050786227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002699654398838902) - actor/ppo_kl:np.float64(0.00015283571616642178) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(3888.0846324316226) - self_distillation/token_reweight_w_std:np.float64(94961.07842942793) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017440444789827) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060424512790632434) - self_distillation/empty_target_batch:np.float64(0.078125) - actor/grad_norm:np.float64(0.18024816177785397) - perf/mfu/actor:np.float64(0.052331017405947575) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7208366394043) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007848478853702545 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007848478853702545 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.671875 - response_length/max:1662.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.671875 - response_length_non_aborted/max:1662.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.252876043319702e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9589528422802687) - timing_s/agent_loop/generate_sequences/max:np.float64(12.19803000986576) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.432029293398955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.19803000986576) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1662 - timing_s/gen:19.05821064300835 - timing_s/reward:0.0605444461107254 - timing_s/old_log_prob:3.044057449325919 - timing_s/adv:0.641364049166441 - timing_s/update_actor:22.09057642146945 - timing_s/step:44.98519729450345 - timing_s/stop_profile:3.0197203159332275e-05 - timing_per_token_ms/gen:0.17009577168798284 - timing_per_token_ms/update_actor:0.12488736359122052 - timing_per_token_ms/adv:0.0036259019988605016 - perf/total_num_tokens:176884 - perf/time_per_step:44.98519729450345 - perf/throughput:491.50612489814705 (TaskRunner pid=2320396) Training Progress: 56%|█████▌ | 56/100 [54:19<35:52, 48.92s/it] (TaskRunner pid=2320396) step:57 - global_seqlen/min:20660 - global_seqlen/max:27723 - global_seqlen/minmax_diff:7063 - global_seqlen/balanced_min:24017 - global_seqlen/balanced_max:24026 - global_seqlen/mean:24022.5 - actor/entropy:0.21244722604751587 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48743531107902527 - training/rollout_probs_diff_mean:0.0028552731964737177 - training/rollout_probs_diff_std:0.009358067996799946 - training/rollout_actor_probs_pearson_corr:0.9991409778594971 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.999923825263977 - rollout_corr/rollout_is_ratio_fraction_high:8.303716640511993e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4911150831030682e-05 - rollout_corr/rollout_is_max:2.55444073677063 - rollout_corr/rollout_is_min:0.3948735296726227 - rollout_corr/rollout_is_std:0.028060244396328926 - rollout_corr/rollout_is_eff_sample_size:0.9992131230960879 - rollout_corr/rollout_is_seq_mean:0.9999536871910095 - rollout_corr/rollout_is_seq_std:0.001707185059785843 - rollout_corr/rollout_is_seq_max:1.0068445205688477 - rollout_corr/rollout_is_seq_min:0.9935453534126282 - rollout_corr/rollout_is_seq_max_deviation:0.006844520568847656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2550159809179604) - rollout_corr/training_log_ppl:np.float64(0.21568975165428128) - rollout_corr/kl:np.float64(0.0006072405742267861) - rollout_corr/k3_kl:np.float64(0.0005212998507886368) - rollout_corr/rollout_ppl:np.float64(1.2542184651829302) - rollout_corr/rollout_log_ppl:np.float64(0.2150825107237324) - rollout_corr/log_ppl_diff:np.float64(0.0006072409305488691) - rollout_corr/log_ppl_abs_diff:np.float64(0.001481423867517151) - rollout_corr/log_ppl_diff_max:np.float64(0.010948315262794495) - rollout_corr/log_ppl_diff_min:np.float64(-0.006019413471221924) - rollout_corr/ppl_ratio:np.float64(1.0006093410775065) - rollout_corr/chi2_token:np.float64(0.0008280288893729448) - rollout_corr/chi2_seq:np.float64(0.5540966941043735) - actor/pg_loss:np.float64(0.001621344476006925) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026958688931699726) - actor/ppo_kl:np.float64(0.000122946722594941) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(1276.8657940202393) - self_distillation/token_reweight_w_std:np.float64(25753.365622778947) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988068183884025) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05870081028842833) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.16179810464382172) - perf/mfu/actor:np.float64(0.05671102563507794) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.77534103393555) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032415634486824274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032415634486824274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.421875 - response_length/max:1224.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.421875 - response_length_non_aborted/max:1224.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.60230153799057e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7831499483436346) - timing_s/agent_loop/generate_sequences/max:np.float64(10.127770842984319) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.204904470600013) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.127770842984319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1224 - timing_s/gen:18.107756910845637 - timing_s/reward:0.05700390413403511 - timing_s/old_log_prob:3.0384509321302176 - timing_s/adv:0.5749542769044638 - timing_s/update_actor:22.06851520948112 - timing_s/step:43.88189716823399 - timing_s/stop_profile:3.085285425186157e-05 - timing_per_token_ms/gen:0.15036168424988905 - timing_per_token_ms/update_actor:0.11483252788781934 - timing_per_token_ms/adv:0.002991748761080569 - perf/total_num_tokens:192180 - perf/time_per_step:43.88189716823399 - perf/throughput:547.4353104630543 (TaskRunner pid=2320396) Training Progress: 57%|█████▋ | 57/100 [55:03<33:59, 47.42s/it] (TaskRunner pid=2320396) step:58 - global_seqlen/min:15671 - global_seqlen/max:26285 - global_seqlen/minmax_diff:10614 - global_seqlen/balanced_min:22595 - global_seqlen/balanced_max:22603 - global_seqlen/mean:22600.5 - actor/entropy:0.2078225016593933 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647239685058594 - training/rollout_probs_diff_mean:0.002780613023787737 - training/rollout_probs_diff_std:0.009222094900906086 - training/rollout_actor_probs_pearson_corr:0.9990860223770142 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9997680187225342 - rollout_corr/rollout_is_ratio_fraction_high:1.7986258171731606e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.697938725759741e-05 - rollout_corr/rollout_is_max:2.3788044452667236 - rollout_corr/rollout_is_min:0.22833219170570374 - rollout_corr/rollout_is_std:0.028010282665491104 - rollout_corr/rollout_is_eff_sample_size:0.9992158606040054 - rollout_corr/rollout_is_seq_mean:0.9997212886810303 - rollout_corr/rollout_is_seq_std:0.001959129935130477 - rollout_corr/rollout_is_seq_max:1.0098477602005005 - rollout_corr/rollout_is_seq_min:0.9926185607910156 - rollout_corr/rollout_is_seq_max_deviation:0.009847760200500488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.232383776921779) - rollout_corr/training_log_ppl:np.float64(0.19788227328535868) - rollout_corr/kl:np.float64(0.0007298278635232691) - rollout_corr/k3_kl:np.float64(0.00048722387552402324) - rollout_corr/rollout_ppl:np.float64(1.2314080968499184) - rollout_corr/rollout_log_ppl:np.float64(0.19715244446706492) - rollout_corr/log_ppl_diff:np.float64(0.000729828818293754) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014633310711360537) - rollout_corr/log_ppl_diff_max:np.float64(0.008250519633293152) - rollout_corr/log_ppl_diff_min:np.float64(-0.008565150201320648) - rollout_corr/ppl_ratio:np.float64(1.000732037704438) - rollout_corr/chi2_token:np.float64(0.0004544253461062908) - rollout_corr/chi2_seq:np.float64(0.14290119265206158) - actor/pg_loss:np.float64(0.0023564978037029505) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047837036345299566) - actor/ppo_kl:np.float64(1.843932994916031e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(189.77452703402378) - self_distillation/token_reweight_w_std:np.float64(4119.465276905161) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004234984517097) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06107424577930942) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.18996725231409073) - perf/mfu/actor:np.float64(0.05371021553273776) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.75103378295898) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012600498273968697 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012600498273968697 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.359375 - response_length/max:1079.0 - response_length/min:56.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.359375 - response_length_non_aborted/max:1079.0 - response_length_non_aborted/min:56.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.9674341678619385e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7030046246945858) - timing_s/agent_loop/generate_sequences/max:np.float64(9.170371565967798) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.49670002021594) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.170371565967798) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1079 - timing_s/gen:15.731107097119093 - timing_s/reward:0.06631920486688614 - timing_s/old_log_prob:3.030934613198042 - timing_s/adv:0.5863243769854307 - timing_s/update_actor:22.018998054787517 - timing_s/step:41.53536947630346 - timing_s/stop_profile:0.00011384859681129456 - timing_per_token_ms/gen:0.14147187935824215 - timing_per_token_ms/update_actor:0.12178379933401648 - timing_per_token_ms/adv:0.0032428728179986657 - perf/total_num_tokens:180804 - perf/time_per_step:41.53536947630346 - perf/throughput:544.1266150983421 (TaskRunner pid=2320396) Training Progress: 58%|█████▊ | 58/100 [55:45<31:58, 45.67s/it] (TaskRunner pid=2320396) step:59 - global_seqlen/min:16079 - global_seqlen/max:27248 - global_seqlen/minmax_diff:11169 - global_seqlen/balanced_min:22015 - global_seqlen/balanced_max:22027 - global_seqlen/mean:22023.625 - actor/entropy:0.1970500946044922 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20963798463344574 - training/rollout_probs_diff_mean:0.002791076432913542 - training/rollout_probs_diff_std:0.009237607941031456 - training/rollout_actor_probs_pearson_corr:0.9990670084953308 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.000091314315796 - rollout_corr/rollout_is_ratio_fraction_high:9.29601264942903e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.29601264942903e-06 - rollout_corr/rollout_is_max:2.8204736709594727 - rollout_corr/rollout_is_min:0.44322657585144043 - rollout_corr/rollout_is_std:0.028172601014375687 - rollout_corr/rollout_is_eff_sample_size:0.9992070530233411 - rollout_corr/rollout_is_seq_mean:0.9999852180480957 - rollout_corr/rollout_is_seq_std:0.0020917868241667747 - rollout_corr/rollout_is_seq_max:1.011966347694397 - rollout_corr/rollout_is_seq_min:0.9925203323364258 - rollout_corr/rollout_is_seq_max_deviation:0.011966347694396973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2375959348864853) - rollout_corr/training_log_ppl:np.float64(0.19846132550446782) - rollout_corr/kl:np.float64(0.000528362165669094) - rollout_corr/k3_kl:np.float64(0.0005359913304374686) - rollout_corr/rollout_ppl:np.float64(1.2368552833795547) - rollout_corr/rollout_log_ppl:np.float64(0.19793296113493852) - rollout_corr/log_ppl_diff:np.float64(0.0005283643695292994) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013686856691492721) - rollout_corr/log_ppl_diff_max:np.float64(0.01003563404083252) - rollout_corr/log_ppl_diff_min:np.float64(-0.009744033217430115) - rollout_corr/ppl_ratio:np.float64(1.0005305462982506) - rollout_corr/chi2_token:np.float64(0.0012348862364888191) - rollout_corr/chi2_seq:np.float64(0.18860497209243476) - actor/pg_loss:np.float64(0.0033428192837163806) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003301967922197946) - actor/ppo_kl:np.float64(7.350512616555172e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(5227.270080071641) - self_distillation/token_reweight_w_std:np.float64(87981.93152151996) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998072686139494) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06207750209432561) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.25658440589904785) - perf/mfu/actor:np.float64(0.05250423204988193) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7232894897461) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007577412761747837 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007577412761747837 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.20703125 - response_length/max:1145.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.20703125 - response_length_non_aborted/max:1145.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014202482998371124 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9425349365919828) - timing_s/agent_loop/generate_sequences/max:np.float64(9.21257615275681) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.516783446379122) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.21257615275681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1145 - timing_s/gen:16.408081464469433 - timing_s/reward:0.06142246536910534 - timing_s/old_log_prob:3.069812884554267 - timing_s/adv:0.5685435552150011 - timing_s/update_actor:21.84132767468691 - timing_s/step:42.037929989397526 - timing_s/stop_profile:0.00012028217315673828 - timing_per_token_ms/gen:0.1525297376151026 - timing_per_token_ms/update_actor:0.12396533083613 - timing_per_token_ms/adv:0.0032268958630504805 - perf/total_num_tokens:176189 - perf/time_per_step:42.037929989397526 - perf/throughput:523.8988933459527 (TaskRunner pid=2320396) Training Progress: 59%|█████▉ | 59/100 [56:27<30:28, 44.59s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rounding to the nearest whole number, the magnitude of the charge is approximately $ 56 \, \mathrm{pC} $. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2320396) step:60 - global_seqlen/min:16861 - global_seqlen/max:28185 - global_seqlen/minmax_diff:11324 - global_seqlen/balanced_min:22957 - global_seqlen/balanced_max:22976 - global_seqlen/mean:22970.875 - actor/entropy:0.20119284093379974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41020888090133667 - training/rollout_probs_diff_mean:0.0027175727300345898 - training/rollout_probs_diff_std:0.00927771907299757 - training/rollout_actor_probs_pearson_corr:0.9991341829299927 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0001078844070435 - rollout_corr/rollout_is_ratio_fraction_high:2.6176410756306723e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7450940504204482e-05 - rollout_corr/rollout_is_max:2.1670923233032227 - rollout_corr/rollout_is_min:0.39598017930984497 - rollout_corr/rollout_is_std:0.028364475816488266 - rollout_corr/rollout_is_eff_sample_size:0.999196341310165 - rollout_corr/rollout_is_seq_mean:1.0001099109649658 - rollout_corr/rollout_is_seq_std:0.0016272783977910876 - rollout_corr/rollout_is_seq_max:1.0065743923187256 - rollout_corr/rollout_is_seq_min:0.9956563115119934 - rollout_corr/rollout_is_seq_max_deviation:0.006574392318725586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2329197567887604) - rollout_corr/training_log_ppl:np.float64(0.19494881299033295) - rollout_corr/kl:np.float64(0.0005096376898059418) - rollout_corr/k3_kl:np.float64(0.000489828710598772) - rollout_corr/rollout_ppl:np.float64(1.2322118538431823) - rollout_corr/rollout_log_ppl:np.float64(0.19443917492753826) - rollout_corr/log_ppl_diff:np.float64(0.0005096380627946928) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013880194601370022) - rollout_corr/log_ppl_diff_max:np.float64(0.0056622326374053955) - rollout_corr/log_ppl_diff_min:np.float64(-0.005925208330154419) - rollout_corr/ppl_ratio:np.float64(1.000511354766786) - rollout_corr/chi2_token:np.float64(0.00095889694057405) - rollout_corr/chi2_seq:np.float64(0.9751317095942795) - actor/pg_loss:np.float64(0.001680221175774932) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000293630393116473) - actor/ppo_kl:np.float64(0.00020125376903123993) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(45.16269864095375) - self_distillation/token_reweight_w_std:np.float64(918.4515415658243) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998249302152544) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03986541925405618) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.17679162323474884) - perf/mfu/actor:np.float64(0.054896138579939024) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.72972869873047) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1910811494167271) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7283999999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15493982751146623) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.561325) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1694866363964564) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6451) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19198155285050436) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7913375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10282284928700713) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49381249999999993) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12633365609527897) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.66425) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16169326315203264) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8276625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05186472248141516) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4475625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08316721871278766) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6774875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12662312734272863) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8432000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.019071806002745252) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41564999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05142809598951441) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6827375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09880717472092095) - val-aux/sciknoweval/score/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/score/std@16:np.float64(0.1910811494167271) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7283999999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15493982751146623) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.561325) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1694866363964564) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6451) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19198155285050436) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7913375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10282284928700713) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49381249999999993) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12633365609527897) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.66425) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16169326315203264) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8276625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05186472248141516) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4475625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08316721871278766) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6774875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12662312734272863) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8432000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.019071806002745252) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41564999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05142809598951441) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6827375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09880717472092095) - val-core/sciknoweval/acc/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1910811494167271) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7283999999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15493982751146623) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.561325) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1694866363964564) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6451) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19198155285050436) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7913375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10282284928700713) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49381249999999993) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12633365609527897) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.66425) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16169326315203264) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8276625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05186472248141516) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4475625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08316721871278766) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6774875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12662312734272863) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8432000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.019071806002745252) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41564999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05142809598951441) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6827375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09880717472092095) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98671875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.030342711941217525) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9966250000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013654021672028597) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.976625) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03787881465318859) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9869125000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.030202946000179105) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995125) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0027850399441387277) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.962275) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04219717155049714) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9939125000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.020819912791213174) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0003950870157319778) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9445375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04278961022314208) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9984000000000002) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010747648297244363) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9239875000000002) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0361678881045997) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99985) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0021922653339917476) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005840611644089222 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005840611644089222 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:447.68359375 - response_length/max:1305.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:447.68359375 - response_length_non_aborted/max:1305.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.737614214420319e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5804355461150408) - timing_s/agent_loop/generate_sequences/max:np.float64(10.886121707037091) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.964501895803551) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.886121707037091) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:1305 - timing_s/gen:17.71578363329172 - timing_s/reward:0.07900326326489449 - timing_s/old_log_prob:3.0846715457737446 - timing_s/adv:0.5956523399800062 - timing_s/update_actor:21.899688495323062 - timing_s/step:43.44701548665762 - timing_s/testing:107.27109943144023 - timing_s/save_checkpoint:12.707542151212692 - timing_s/stop_profile:0.00014159828424453735 - timing_per_token_ms/gen:0.154578547848663 - timing_per_token_ms/update_actor:0.11917095286598281 - timing_per_token_ms/adv:0.003241345508061873 - perf/total_num_tokens:183767 - perf/time_per_step:43.44701548665762 - perf/throughput:528.7100792240666 (TaskRunner pid=2320396) Training Progress: 60%|██████ | 60/100 [59:10<53:30, 80.27s/it] (TaskRunner pid=2320396) step:61 - global_seqlen/min:19183 - global_seqlen/max:30554 - global_seqlen/minmax_diff:11371 - global_seqlen/balanced_min:24395 - global_seqlen/balanced_max:24418 - global_seqlen/mean:24411.25 - actor/entropy:0.20756813883781433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3475205600261688 - training/rollout_probs_diff_mean:0.0026950256433337927 - training/rollout_probs_diff_std:0.009178821928799152 - training/rollout_actor_probs_pearson_corr:0.9992281794548035 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999772906303406 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5400490156025626e-05 - rollout_corr/rollout_is_max:1.9424793720245361 - rollout_corr/rollout_is_min:0.32020506262779236 - rollout_corr/rollout_is_std:0.02756519615650177 - rollout_corr/rollout_is_eff_sample_size:0.9992406178785591 - rollout_corr/rollout_is_seq_mean:0.9998874664306641 - rollout_corr/rollout_is_seq_std:0.0018777569057419896 - rollout_corr/rollout_is_seq_max:1.0054404735565186 - rollout_corr/rollout_is_seq_min:0.9923277497291565 - rollout_corr/rollout_is_seq_max_deviation:0.007672250270843506 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2361338967457414) - rollout_corr/training_log_ppl:np.float64(0.19852098800765816) - rollout_corr/kl:np.float64(0.0005635680740212479) - rollout_corr/k3_kl:np.float64(0.00048129173163147243) - rollout_corr/rollout_ppl:np.float64(1.2354488540440798) - rollout_corr/rollout_log_ppl:np.float64(0.1979574196448084) - rollout_corr/log_ppl_diff:np.float64(0.0005635683628497645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013617147196782753) - rollout_corr/log_ppl_diff_max:np.float64(0.009035363793373108) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046346187591552734) - rollout_corr/ppl_ratio:np.float64(1.0005656315479428) - rollout_corr/chi2_token:np.float64(0.0008184912148863077) - rollout_corr/chi2_seq:np.float64(0.38796538999304175) - actor/pg_loss:np.float64(0.0034785300958901644) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003513928604661487) - actor/ppo_kl:np.float64(2.1584473852698238e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(21164.804104407784) - self_distillation/token_reweight_w_std:np.float64(310615.7949758964) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992343427147716) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06176384340506047) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.20081432536244392) - perf/mfu/actor:np.float64(0.056794247237666955) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.04122829437256) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004993608687072992 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004993608687072992 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:507.2890625 - response_length/max:1599.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:507.2890625 - response_length_non_aborted/max:1599.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015490874648094177 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9240107480436563) - timing_s/agent_loop/generate_sequences/max:np.float64(12.865415262058377) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.402125497661473) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.865415262058377) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1599 - timing_s/gen:19.278599567711353 - timing_s/reward:0.06761794164776802 - timing_s/old_log_prob:3.1490181647241116 - timing_s/adv:0.639240212738514 - timing_s/update_actor:22.43888548016548 - timing_s/step:45.67836303450167 - timing_s/stop_profile:0.00011006183922290802 - timing_per_token_ms/gen:0.14844993737938608 - timing_per_token_ms/update_actor:0.1149003301764836 - timing_per_token_ms/adv:0.003273286971880352 - perf/total_num_tokens:195290 - perf/time_per_step:45.67836303450167 - perf/throughput:534.4160424829969 (TaskRunner pid=2320396) Training Progress: 61%|██████ | 61/100 [59:56<45:26, 69.91s/it] (TaskRunner pid=2320396) step:62 - global_seqlen/min:20758 - global_seqlen/max:30529 - global_seqlen/minmax_diff:9771 - global_seqlen/balanced_min:25475 - global_seqlen/balanced_max:25505 - global_seqlen/mean:25489.375 - actor/entropy:0.23646610975265503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163541436195374 - training/rollout_probs_diff_mean:0.0028907926753163338 - training/rollout_probs_diff_std:0.009120183065533638 - training/rollout_actor_probs_pearson_corr:0.9992669224739075 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000348091125488 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.682202522119042e-06 - rollout_corr/rollout_is_max:1.6338132619857788 - rollout_corr/rollout_is_min:0.04254038631916046 - rollout_corr/rollout_is_std:0.027586810290813446 - rollout_corr/rollout_is_eff_sample_size:0.9992395466249402 - rollout_corr/rollout_is_seq_mean:1.0000940561294556 - rollout_corr/rollout_is_seq_std:0.0016261922428384423 - rollout_corr/rollout_is_seq_max:1.0076667070388794 - rollout_corr/rollout_is_seq_min:0.9946310520172119 - rollout_corr/rollout_is_seq_max_deviation:0.0076667070388793945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2761757029220462) - rollout_corr/training_log_ppl:np.float64(0.22961067700816784) - rollout_corr/kl:np.float64(0.0003324805734532532) - rollout_corr/k3_kl:np.float64(0.00043630713193465454) - rollout_corr/rollout_ppl:np.float64(1.275736337993294) - rollout_corr/rollout_log_ppl:np.float64(0.229278196071391) - rollout_corr/log_ppl_diff:np.float64(0.00033248093677684665) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012090119707863778) - rollout_corr/log_ppl_diff_max:np.float64(0.006240293383598328) - rollout_corr/log_ppl_diff_min:np.float64(-0.007163435220718384) - rollout_corr/ppl_ratio:np.float64(1.0003339080139995) - rollout_corr/chi2_token:np.float64(0.0010929901618510485) - rollout_corr/chi2_seq:np.float64(0.3629838922061026) - actor/pg_loss:np.float64(0.0025408605579286814) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021029063645983115) - actor/ppo_kl:np.float64(2.6390286969046883e-06) - actor/pg_clipfrac_lower:np.float64(4.116174750379287e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(9723.37822908233) - self_distillation/token_reweight_w_std:np.float64(233810.96705805685) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999671149533242) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0595514630258549) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.18411173671483994) - perf/mfu/actor:np.float64(0.058899028442019795) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.08804321289062) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028481765184551477 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028481765184551477 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:508.48046875 - response_length/max:1441.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:508.48046875 - response_length_non_aborted/max:1441.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.119642734527588e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0143952444195747) - timing_s/agent_loop/generate_sequences/max:np.float64(11.363589746877551) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.538877304374182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.363589746877551) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1379 - timing_s/gen:18.385584201663733 - timing_s/reward:0.06554795429110527 - timing_s/old_log_prob:3.051215363666415 - timing_s/adv:0.6370475739240646 - timing_s/update_actor:22.51959772594273 - timing_s/step:44.75173839740455 - timing_s/stop_profile:0.00012248754501342773 - timing_per_token_ms/gen:0.1412417835129463 - timing_per_token_ms/update_actor:0.11043620001443116 - timing_per_token_ms/adv:0.0031240839267541115 - perf/total_num_tokens:203915 - perf/time_per_step:44.75173839740455 - perf/throughput:569.5728459450929 (TaskRunner pid=2320396) Training Progress: 62%|██████▏ | 62/100 [1:00:41<39:30, 62.37s/it] (TaskRunner pid=2320396) step:63 - global_seqlen/min:18323 - global_seqlen/max:25373 - global_seqlen/minmax_diff:7050 - global_seqlen/balanced_min:22190 - global_seqlen/balanced_max:22198 - global_seqlen/mean:22195.25 - actor/entropy:0.22693386673927307 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47528380155563354 - training/rollout_probs_diff_mean:0.0029311031103134155 - training/rollout_probs_diff_std:0.009294695220887661 - training/rollout_actor_probs_pearson_corr:0.9991291165351868 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.000087022781372 - rollout_corr/rollout_is_ratio_fraction_high:1.8302614989806898e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.348695755004883 - rollout_corr/rollout_is_min:0.5427963733673096 - rollout_corr/rollout_is_std:0.02828240394592285 - rollout_corr/rollout_is_eff_sample_size:0.9992008640054926 - rollout_corr/rollout_is_seq_mean:1.0000746250152588 - rollout_corr/rollout_is_seq_std:0.001608627731911838 - rollout_corr/rollout_is_seq_max:1.0082793235778809 - rollout_corr/rollout_is_seq_min:0.9936819672584534 - rollout_corr/rollout_is_seq_max_deviation:0.00827932357788086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2453211299143732) - rollout_corr/training_log_ppl:np.float64(0.2045088292506989) - rollout_corr/kl:np.float64(0.00037927413301552804) - rollout_corr/k3_kl:np.float64(0.0004077912325044508) - rollout_corr/rollout_ppl:np.float64(1.2448103982023895) - rollout_corr/rollout_log_ppl:np.float64(0.204129554957035) - rollout_corr/log_ppl_diff:np.float64(0.0003792742936639115) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012008958583464846) - rollout_corr/log_ppl_diff_max:np.float64(0.005300462245941162) - rollout_corr/log_ppl_diff_min:np.float64(-0.009547919034957886) - rollout_corr/ppl_ratio:np.float64(1.0003806757740676) - rollout_corr/chi2_token:np.float64(0.0008865287527441978) - rollout_corr/chi2_seq:np.float64(0.3732549585402012) - actor/pg_loss:np.float64(0.001522935344837606) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011153514515171992) - actor/ppo_kl:np.float64(4.508457355001383e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(3185.1990010682493) - self_distillation/token_reweight_w_std:np.float64(66056.73461075983) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00030661188066) - self_distillation/token_reweight_w_clip_frac:np.float64(0.032705511490348727) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.1341842170804739) - perf/mfu/actor:np.float64(0.05382390301504096) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.0361328125) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006129088345915079 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006129088345915079 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:426.8515625 - response_length/max:1065.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.8515625 - response_length_non_aborted/max:1065.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015802867710590363 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8849795944988728) - timing_s/agent_loop/generate_sequences/max:np.float64(9.163215219974518) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.294443838567531) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.163215219974518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:1065 - timing_s/gen:15.782489329576492 - timing_s/reward:0.07164940424263477 - timing_s/old_log_prob:3.163828060030937 - timing_s/adv:0.542920557782054 - timing_s/update_actor:21.563510172069073 - timing_s/step:41.211437748745084 - timing_s/stop_profile:0.0001256987452507019 - timing_per_token_ms/gen:0.14443041647213878 - timing_per_token_ms/update_actor:0.12144214512152979 - timing_per_token_ms/adv:0.0030576393472818166 - perf/total_num_tokens:177562 - perf/time_per_step:41.211437748745084 - perf/throughput:538.5701449029368 (TaskRunner pid=2320396) Training Progress: 63%|██████▎ | 63/100 [1:01:22<34:33, 56.04s/it] (TaskRunner pid=2320396) step:64 - global_seqlen/min:19205 - global_seqlen/max:31322 - global_seqlen/minmax_diff:12117 - global_seqlen/balanced_min:24735 - global_seqlen/balanced_max:24761 - global_seqlen/mean:24748.75 - actor/entropy:0.23320099711418152 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835739970207214 - training/rollout_probs_diff_mean:0.002753478242084384 - training/rollout_probs_diff_std:0.008778145536780357 - training/rollout_actor_probs_pearson_corr:0.9993224143981934 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999129772186279 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.705703865212854e-06 - rollout_corr/rollout_is_max:1.6222718954086304 - rollout_corr/rollout_is_min:0.47236648201942444 - rollout_corr/rollout_is_std:0.02701038494706154 - rollout_corr/rollout_is_eff_sample_size:0.9992709710189538 - rollout_corr/rollout_is_seq_mean:0.9999531507492065 - rollout_corr/rollout_is_seq_std:0.0013188592856749892 - rollout_corr/rollout_is_seq_max:1.0045053958892822 - rollout_corr/rollout_is_seq_min:0.9948081970214844 - rollout_corr/rollout_is_seq_max_deviation:0.005191802978515625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2561526354402304) - rollout_corr/training_log_ppl:np.float64(0.212319816739182) - rollout_corr/kl:np.float64(0.00036533900832669275) - rollout_corr/k3_kl:np.float64(0.000400368980734811) - rollout_corr/rollout_ppl:np.float64(1.255633560474962) - rollout_corr/rollout_log_ppl:np.float64(0.21195447669015266) - rollout_corr/log_ppl_diff:np.float64(0.00036534004902932793) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011143776500830427) - rollout_corr/log_ppl_diff_max:np.float64(0.005632832646369934) - rollout_corr/log_ppl_diff_min:np.float64(-0.005937211215496063) - rollout_corr/ppl_ratio:np.float64(1.0003664672840387) - rollout_corr/chi2_token:np.float64(0.0008791075088083744) - rollout_corr/chi2_seq:np.float64(0.31506598158739507) - actor/pg_loss:np.float64(0.0030615170253440738) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000200204512111668) - actor/ppo_kl:np.float64(-5.050095456127224e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(12580.71558534773) - self_distillation/token_reweight_w_std:np.float64(320267.26721710706) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003797668032348) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05963162177067716) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.16736289486289024) - perf/mfu/actor:np.float64(0.05846823715557507) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.96532821655273) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009651394211687148 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009651394211687148 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:506.9296875 - response_length/max:1500.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:506.9296875 - response_length_non_aborted/max:1500.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.725987911224365e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8472258783876896) - timing_s/agent_loop/generate_sequences/max:np.float64(12.097647780552506) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.7089943648461485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.097647780552506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:349 - timing_s/agent_loop/slowest/response_length:1500 - timing_s/gen:18.741990983486176 - timing_s/reward:0.06568616442382336 - timing_s/old_log_prob:3.0562352687120438 - timing_s/adv:0.6292859483510256 - timing_s/update_actor:22.034651696681976 - timing_s/step:44.61622857116163 - timing_s/stop_profile:0.00011373311281204224 - timing_per_token_ms/gen:0.14442023042740593 - timing_per_token_ms/update_actor:0.1112917404751855 - timing_per_token_ms/adv:0.0031783723842165037 - perf/total_num_tokens:197990 - perf/time_per_step:44.61622857116163 - perf/throughput:554.7028691707199 (TaskRunner pid=2320396) Training Progress: 64%|██████▍ | 64/100 [1:02:07<31:34, 52.63s/it] (TaskRunner pid=2320396) step:65 - global_seqlen/min:12805 - global_seqlen/max:27893 - global_seqlen/minmax_diff:15088 - global_seqlen/balanced_min:21975 - global_seqlen/balanced_max:21991 - global_seqlen/mean:21986.0 - actor/entropy:0.19277343153953552 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135476350784302 - training/rollout_probs_diff_mean:0.002630578586831689 - training/rollout_probs_diff_std:0.008825007826089859 - training/rollout_actor_probs_pearson_corr:0.9991601705551147 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.84375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:1.0000410079956055 - rollout_corr/rollout_is_ratio_fraction_high:9.218969353241846e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8437938706483692e-05 - rollout_corr/rollout_is_max:2.167001485824585 - rollout_corr/rollout_is_min:0.4516155421733856 - rollout_corr/rollout_is_std:0.02641010284423828 - rollout_corr/rollout_is_eff_sample_size:0.9993031116481851 - rollout_corr/rollout_is_seq_mean:1.000004768371582 - rollout_corr/rollout_is_seq_std:0.0014547642786055803 - rollout_corr/rollout_is_seq_max:1.0044742822647095 - rollout_corr/rollout_is_seq_min:0.9930375218391418 - rollout_corr/rollout_is_seq_max_deviation:0.006962478160858154 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.188718719407916) - rollout_corr/training_log_ppl:np.float64(0.16565465025632875) - rollout_corr/kl:np.float64(0.0005832098385161189) - rollout_corr/k3_kl:np.float64(0.0003921816246865717) - rollout_corr/rollout_ppl:np.float64(1.1879985383711755) - rollout_corr/rollout_log_ppl:np.float64(0.16507143871422159) - rollout_corr/log_ppl_diff:np.float64(0.0005832115421071649) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012660615611821413) - rollout_corr/log_ppl_diff_max:np.float64(0.012264132499694824) - rollout_corr/log_ppl_diff_min:np.float64(-0.003919497132301331) - rollout_corr/ppl_ratio:np.float64(1.0005849439185113) - rollout_corr/chi2_token:np.float64(0.0003987816162407398) - rollout_corr/chi2_seq:np.float64(0.1664372819941491) - actor/pg_loss:np.float64(0.0017755826702341437) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001691463867246057) - actor/ppo_kl:np.float64(0.00022979352656449237) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_w_mean:np.float64(11.275148867396638) - self_distillation/token_reweight_w_std:np.float64(238.43248290149495) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994188658893108) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03806430572876707) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.10744893923401833) - perf/mfu/actor:np.float64(0.05209502512150299) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.95738220214844) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.84375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008053922094404697 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008053922094404697 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.71875 - response_length/max:1412.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.71875 - response_length_non_aborted/max:1412.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001363661140203476 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9663147907704115) - timing_s/agent_loop/generate_sequences/max:np.float64(10.955320874229074) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.436157472024206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.955320874229074) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:363 - timing_s/agent_loop/slowest/response_length:1412 - timing_s/gen:18.540402602404356 - timing_s/reward:0.061292244121432304 - timing_s/old_log_prob:3.1427970565855503 - timing_s/adv:0.5885716322809458 - timing_s/update_actor:21.8789913803339 - timing_s/step:44.30024716630578 - timing_s/stop_profile:0.00010786019265651703 - timing_per_token_ms/gen:0.1709233959215683 - timing_per_token_ms/update_actor:0.12439160932146537 - timing_per_token_ms/adv:0.003346286456614128 - perf/total_num_tokens:175888 - perf/time_per_step:44.30024716630578 - perf/throughput:496.29519938033843 (TaskRunner pid=2320396) Training Progress: 65%|██████▌ | 65/100 [1:02:51<29:15, 50.14s/it] (TaskRunner pid=2320396) step:66 - global_seqlen/min:21157 - global_seqlen/max:30988 - global_seqlen/minmax_diff:9831 - global_seqlen/balanced_min:24722 - global_seqlen/balanced_max:24738 - global_seqlen/mean:24733.625 - actor/entropy:0.22358284890651703 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413550317287445 - training/rollout_probs_diff_mean:0.002847072435542941 - training/rollout_probs_diff_std:0.00899769738316536 - training/rollout_actor_probs_pearson_corr:0.9991993308067322 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.000038504600525 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.874790906906128 - rollout_corr/rollout_is_min:0.5706772804260254 - rollout_corr/rollout_is_std:0.027571681886911392 - rollout_corr/rollout_is_eff_sample_size:0.9992406178785591 - rollout_corr/rollout_is_seq_mean:0.999980628490448 - rollout_corr/rollout_is_seq_std:0.0014468131121248007 - rollout_corr/rollout_is_seq_max:1.0045393705368042 - rollout_corr/rollout_is_seq_min:0.9940057396888733 - rollout_corr/rollout_is_seq_max_deviation:0.005994260311126709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.241884677670896) - rollout_corr/training_log_ppl:np.float64(0.20733708786428906) - rollout_corr/kl:np.float64(0.0005020291563688772) - rollout_corr/k3_kl:np.float64(0.0004314393266895422) - rollout_corr/rollout_ppl:np.float64(1.2412674692459404) - rollout_corr/rollout_log_ppl:np.float64(0.20683505745546427) - rollout_corr/log_ppl_diff:np.float64(0.000502030408824794) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011577872064663097) - rollout_corr/log_ppl_diff_max:np.float64(0.006901249289512634) - rollout_corr/log_ppl_diff_min:np.float64(-0.005114339292049408) - rollout_corr/ppl_ratio:np.float64(1.0005033700726926) - rollout_corr/chi2_token:np.float64(0.0007429802790284157) - rollout_corr/chi2_seq:np.float64(0.2530661830678582) - actor/pg_loss:np.float64(0.0027268020203337073) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020032282054671668) - actor/ppo_kl:np.float64(9.502547750450674e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(5488.480364457937) - self_distillation/token_reweight_w_std:np.float64(106593.19712470478) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994499702006578) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06123181653674692) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.1668576430529356) - perf/mfu/actor:np.float64(0.05822141794440938) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.87117385864258) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008164207451045513 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008164207451045513 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:493.11328125 - response_length/max:1246.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:493.11328125 - response_length_non_aborted/max:1246.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001009814441204071 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1982299461960793) - timing_s/agent_loop/generate_sequences/max:np.float64(10.634438088163733) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.621325811313) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.634438088163733) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1246 - timing_s/gen:17.873955199494958 - timing_s/reward:0.06542827747762203 - timing_s/old_log_prob:3.105355206876993 - timing_s/adv:0.5925860479474068 - timing_s/update_actor:22.105792071670294 - timing_s/step:43.8312210701406 - timing_s/stop_profile:0.00012839213013648987 - timing_per_token_ms/gen:0.14159046238024475 - timing_per_token_ms/update_actor:0.11171932981755754 - timing_per_token_ms/adv:0.0029948402627364913 - perf/total_num_tokens:197869 - perf/time_per_step:43.8312210701406 - perf/throughput:564.292401537712 (TaskRunner pid=2320396) Training Progress: 66%|██████▌ | 66/100 [1:03:35<27:21, 48.27s/it] (TaskRunner pid=2320396) step:67 - global_seqlen/min:12971 - global_seqlen/max:31231 - global_seqlen/minmax_diff:18260 - global_seqlen/balanced_min:21701 - global_seqlen/balanced_max:21708 - global_seqlen/mean:21704.75 - actor/entropy:0.2337634563446045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23616445064544678 - training/rollout_probs_diff_mean:0.002843605587258935 - training/rollout_probs_diff_std:0.008973178453743458 - training/rollout_actor_probs_pearson_corr:0.9992346167564392 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999405741691589 - rollout_corr/rollout_is_ratio_fraction_high:9.46414002100937e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.378192901611328 - rollout_corr/rollout_is_min:0.5167847275733948 - rollout_corr/rollout_is_std:0.027302302420139313 - rollout_corr/rollout_is_eff_sample_size:0.999254901479631 - rollout_corr/rollout_is_seq_mean:0.9998087286949158 - rollout_corr/rollout_is_seq_std:0.0017661856254562736 - rollout_corr/rollout_is_seq_max:1.0052413940429688 - rollout_corr/rollout_is_seq_min:0.9917988777160645 - rollout_corr/rollout_is_seq_max_deviation:0.008201122283935547 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3157822396606207) - rollout_corr/training_log_ppl:np.float64(0.2406519346841378) - rollout_corr/kl:np.float64(0.0006637728164324841) - rollout_corr/k3_kl:np.float64(0.0004682636699158138) - rollout_corr/rollout_ppl:np.float64(1.3148044827394187) - rollout_corr/rollout_log_ppl:np.float64(0.23998815877712332) - rollout_corr/log_ppl_diff:np.float64(0.0006637759070144966) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014726822410011664) - rollout_corr/log_ppl_diff_max:np.float64(0.010123491287231445) - rollout_corr/log_ppl_diff_min:np.float64(-0.007408738136291504) - rollout_corr/ppl_ratio:np.float64(1.0006661214865744) - rollout_corr/chi2_token:np.float64(0.000572551041841507) - rollout_corr/chi2_seq:np.float64(1.99370268243365) - actor/pg_loss:np.float64(0.0007992174942046404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001359532384412887) - actor/ppo_kl:np.float64(5.996200779279093e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(4817.5678551825695) - self_distillation/token_reweight_w_std:np.float64(95755.82945584616) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006719485390931) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034751594881527126) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.12752852961421013) - perf/mfu/actor:np.float64(0.05151760153021234) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.8641757965088) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015237266197800636 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015237266197800636 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.7421875 - response_length/max:1256.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.7421875 - response_length_non_aborted/max:1256.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005677081644535065 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8790447302162647) - timing_s/agent_loop/generate_sequences/max:np.float64(9.784351877868176) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.49863724395982) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.784351877868176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1199 - timing_s/gen:16.717525171115994 - timing_s/reward:0.05824257992208004 - timing_s/old_log_prob:3.199490273371339 - timing_s/adv:0.6258921548724174 - timing_s/update_actor:21.930379945784807 - timing_s/step:42.64087346568704 - timing_s/stop_profile:3.0914321541786194e-05 - timing_per_token_ms/gen:0.15821700489405835 - timing_per_token_ms/update_actor:0.12629942723243073 - timing_per_token_ms/adv:0.003604580534631921 - perf/total_num_tokens:173638 - perf/time_per_step:42.64087346568704 - perf/throughput:509.0127906846406 (TaskRunner pid=2320396) Training Progress: 67%|██████▋ | 67/100 [1:04:18<25:41, 46.73s/it] (TaskRunner pid=2320396) step:68 - global_seqlen/min:19038 - global_seqlen/max:30016 - global_seqlen/minmax_diff:10978 - global_seqlen/balanced_min:22411 - global_seqlen/balanced_max:22428 - global_seqlen/mean:22421.25 - actor/entropy:0.19149050116539001 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553967833518982 - training/rollout_probs_diff_mean:0.0027065107133239508 - training/rollout_probs_diff_std:0.00906725600361824 - training/rollout_actor_probs_pearson_corr:0.9990512132644653 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.7578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999111890792847 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6474685668945312 - rollout_corr/rollout_is_min:0.5086522102355957 - rollout_corr/rollout_is_std:0.026639310643076897 - rollout_corr/rollout_is_eff_sample_size:0.9992909693897 - rollout_corr/rollout_is_seq_mean:0.9998083710670471 - rollout_corr/rollout_is_seq_std:0.0016350612277165055 - rollout_corr/rollout_is_seq_max:1.0051226615905762 - rollout_corr/rollout_is_seq_min:0.9900025129318237 - rollout_corr/rollout_is_seq_max_deviation:0.00999748706817627 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2052938612177968) - rollout_corr/training_log_ppl:np.float64(0.17810455984727014) - rollout_corr/kl:np.float64(0.0006475814677733283) - rollout_corr/k3_kl:np.float64(0.0004210113900739998) - rollout_corr/rollout_ppl:np.float64(1.204493265133351) - rollout_corr/rollout_log_ppl:np.float64(0.1774569788685767) - rollout_corr/log_ppl_diff:np.float64(0.0006475809786934406) - rollout_corr/log_ppl_abs_diff:np.float64(0.001370340964058414) - rollout_corr/log_ppl_diff_max:np.float64(0.01119665801525116) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046173036098480225) - rollout_corr/ppl_ratio:np.float64(1.0006494410336018) - rollout_corr/chi2_token:np.float64(0.0003732861950993538) - rollout_corr/chi2_seq:np.float64(0.1820344750303775) - actor/pg_loss:np.float64(0.0016421022592112422) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019774771476477326) - actor/ppo_kl:np.float64(6.871426979415673e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_w_mean:np.float64(55.99475798616186) - self_distillation/token_reweight_w_std:np.float64(1029.008185678511) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001285537146032) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04700022886390798) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.21867019683122635) - perf/mfu/actor:np.float64(0.053644370184561974) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7801856994629) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010703998617827892 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010703998617827892 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:435.3203125 - response_length/max:1265.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:435.3203125 - response_length_non_aborted/max:1265.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.28882509469986e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0393309127539396) - timing_s/agent_loop/generate_sequences/max:np.float64(10.404665313661098) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.840117222447589) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.404665313661098) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1265 - timing_s/gen:17.57954941689968 - timing_s/reward:0.061834998428821564 - timing_s/old_log_prob:3.1406824830919504 - timing_s/adv:0.5611031148582697 - timing_s/update_actor:21.558307776227593 - timing_s/step:42.98878578469157 - timing_s/stop_profile:0.00012563355267047882 - timing_per_token_ms/gen:0.15774617663806895 - timing_per_token_ms/update_actor:0.1201890381681864 - timing_per_token_ms/adv:0.0031281881856401275 - perf/total_num_tokens:179370 - perf/time_per_step:42.98878578469157 - perf/throughput:521.5604393270459 (TaskRunner pid=2320396) Training Progress: 68%|██████▊ | 68/100 [1:05:01<24:19, 45.62s/it] (TaskRunner pid=2320396) step:69 - global_seqlen/min:18619 - global_seqlen/max:27775 - global_seqlen/minmax_diff:9156 - global_seqlen/balanced_min:23471 - global_seqlen/balanced_max:23483 - global_seqlen/mean:23479.0 - actor/entropy:0.1945514678955078 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18275725841522217 - training/rollout_probs_diff_mean:0.002515253610908985 - training/rollout_probs_diff_std:0.00857163593173027 - training/rollout_actor_probs_pearson_corr:0.9991545677185059 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.80859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9998260140419006 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.606383204460144 - rollout_corr/rollout_is_min:0.5698836445808411 - rollout_corr/rollout_is_std:0.025290658697485924 - rollout_corr/rollout_is_eff_sample_size:0.9993606128205051 - rollout_corr/rollout_is_seq_mean:0.9998338222503662 - rollout_corr/rollout_is_seq_std:0.0015274977777153254 - rollout_corr/rollout_is_seq_max:1.0084797143936157 - rollout_corr/rollout_is_seq_min:0.9939998984336853 - rollout_corr/rollout_is_seq_max_deviation:0.008479714393615723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2252938458696008) - rollout_corr/training_log_ppl:np.float64(0.19016908761113882) - rollout_corr/kl:np.float64(0.000461935019760773) - rollout_corr/k3_kl:np.float64(0.0003738991998147867) - rollout_corr/rollout_ppl:np.float64(1.2247161795385182) - rollout_corr/rollout_log_ppl:np.float64(0.18970715206523892) - rollout_corr/log_ppl_diff:np.float64(0.00046193554589990526) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011312234710203484) - rollout_corr/log_ppl_diff_max:np.float64(0.006538271903991699) - rollout_corr/log_ppl_diff_min:np.float64(-0.007022887468338013) - rollout_corr/ppl_ratio:np.float64(1.0004631178453565) - rollout_corr/chi2_token:np.float64(0.0005808186251670122) - rollout_corr/chi2_seq:np.float64(0.09659922705031931) - actor/pg_loss:np.float64(0.0018763046246021986) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010173328519158531) - actor/ppo_kl:np.float64(-5.9943213223334624e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_w_mean:np.float64(5404.409017624101) - self_distillation/token_reweight_w_std:np.float64(155274.98073743377) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990867469459772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04784997092792764) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.14408646896481514) - perf/mfu/actor:np.float64(0.055353113989175064) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7887077331543) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011332765221595764 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011332765221595764 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.0 - response_length/max:1312.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.0 - response_length_non_aborted/max:1312.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010467134416103363 - timing_s/agent_loop/generate_sequences/min:np.float64(1.175532167777419) - timing_s/agent_loop/generate_sequences/max:np.float64(10.851345924660563) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.075913681110251) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.851345924660563) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1312 - timing_s/gen:17.613020185381174 - timing_s/reward:0.061091553419828415 - timing_s/old_log_prob:3.0520095359534025 - timing_s/adv:0.5751816276460886 - timing_s/update_actor:22.079808918759227 - timing_s/step:43.46936789341271 - timing_s/stop_profile:0.00011400319635868073 - timing_per_token_ms/gen:0.14859796997655553 - timing_per_token_ms/update_actor:0.11755083755036004 - timing_per_token_ms/adv:0.003062213188626478 - perf/total_num_tokens:187832 - perf/time_per_step:43.46936789341271 - perf/throughput:540.1274768377291 (TaskRunner pid=2320396) Training Progress: 69%|██████▉ | 69/100 [1:05:45<23:14, 44.99s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs ($1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C}$): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2320396) step:70 - global_seqlen/min:18028 - global_seqlen/max:25510 - global_seqlen/minmax_diff:7482 - global_seqlen/balanced_min:22348 - global_seqlen/balanced_max:22365 - global_seqlen/mean:22360.125 - actor/entropy:0.27961182594299316 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4933590888977051 - training/rollout_probs_diff_mean:0.0031229450833052397 - training/rollout_probs_diff_std:0.00926915556192398 - training/rollout_actor_probs_pearson_corr:0.9992610216140747 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.999971330165863 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8122344044968486e-05 - rollout_corr/rollout_is_max:1.530380129814148 - rollout_corr/rollout_is_min:0.4454325735569 - rollout_corr/rollout_is_std:0.02873816154897213 - rollout_corr/rollout_is_eff_sample_size:0.9991745615349487 - rollout_corr/rollout_is_seq_mean:0.9999191164970398 - rollout_corr/rollout_is_seq_std:0.0015768433222547174 - rollout_corr/rollout_is_seq_max:1.0047082901000977 - rollout_corr/rollout_is_seq_min:0.9932003021240234 - rollout_corr/rollout_is_seq_max_deviation:0.0067996978759765625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3106959639117122) - rollout_corr/training_log_ppl:np.float64(0.25108718674164265) - rollout_corr/kl:np.float64(0.0005516339078734944) - rollout_corr/k3_kl:np.float64(0.000464749626843286) - rollout_corr/rollout_ppl:np.float64(1.309946612920612) - rollout_corr/rollout_log_ppl:np.float64(0.25053555284102913) - rollout_corr/log_ppl_diff:np.float64(0.000551633900613524) - rollout_corr/log_ppl_abs_diff:np.float64(0.00132319655676838) - rollout_corr/log_ppl_diff_max:np.float64(0.005755379796028137) - rollout_corr/log_ppl_diff_min:np.float64(-0.005698829889297485) - rollout_corr/ppl_ratio:np.float64(1.0005531306378543) - rollout_corr/chi2_token:np.float64(0.0007509859278798103) - rollout_corr/chi2_seq:np.float64(0.20920691825449467) - actor/pg_loss:np.float64(0.0028931591659784317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023804989859854686) - actor/ppo_kl:np.float64(4.7358733134217346e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(771.5895430163946) - self_distillation/token_reweight_w_std:np.float64(12027.290180076088) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004990971647203) - self_distillation/token_reweight_w_clip_frac:np.float64(0.053546549650491215) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.18996156752109528) - perf/mfu/actor:np.float64(0.05299450687498227) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.74568939208984) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.653125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18547879406462497) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7277750000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1578272523157477) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5789625) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15746179142845634) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6547125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1856290415296593) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7875875000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11695555066473791) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5169125) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12130525026673644) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.66805) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14816289329687846) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.833) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08474610225017794) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4673375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0844809112181804) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6747) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10936134760034488) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8664124999999998) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0552941679483645) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4340625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04729254434864337) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6728999999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07496741716168553) - val-aux/sciknoweval/score/mean@16:np.float64(0.653125) - val-aux/sciknoweval/score/std@16:np.float64(0.18547879406462497) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7277750000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1578272523157477) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5789625) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15746179142845634) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6547125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1856290415296593) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7875875000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11695555066473791) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5169125) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12130525026673644) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.66805) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14816289329687846) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.833) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08474610225017794) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4673375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0844809112181804) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6747) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10936134760034488) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8664124999999998) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0552941679483645) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4340625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04729254434864337) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6728999999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07496741716168553) - val-core/sciknoweval/acc/mean@16:np.float64(0.653125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18547879406462497) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7277750000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1578272523157477) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5789625) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15746179142845634) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6547125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1856290415296593) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7875875000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11695555066473791) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5169125) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12130525026673644) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.66805) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14816289329687846) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.833) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08474610225017794) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4673375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0844809112181804) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6747) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10936134760034488) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8664124999999998) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0552941679483645) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4340625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04729254434864337) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6728999999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07496741716168553) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9953125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0162370593804366) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995499999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.005113892629358849) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9904624999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.022182150680960806) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9950374999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.01666058299867771) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.982925) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02741957965577288) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9982875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.010091735838382115) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9716000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.03053533788791813) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.004125729993095772) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9568249999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.02802487396893062) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010168899782001972 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.010168899782001972 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:431.09765625 - response_length/max:1341.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:431.09765625 - response_length_non_aborted/max:1341.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016302429139614105 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1931192688643932) - timing_s/agent_loop/generate_sequences/max:np.float64(10.98846198990941) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.887176220894617) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.98846198990941) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1341 - timing_s/gen:17.629647402092814 - timing_s/reward:0.06261598318815231 - timing_s/old_log_prob:3.0896036606281996 - timing_s/adv:0.5849062670022249 - timing_s/update_actor:21.95095776952803 - timing_s/step:43.40484565123916 - timing_s/testing:101.84621452912688 - timing_s/save_checkpoint:12.727051159366965 - timing_s/stop_profile:0.00017385371029376984 - timing_per_token_ms/gen:0.15974526691578378 - timing_per_token_ms/update_actor:0.12271262889590304 - timing_per_token_ms/adv:0.0032698065585625357 - perf/total_num_tokens:178881 - perf/time_per_step:43.40484565123916 - perf/throughput:515.1527361637247 (TaskRunner pid=2320396) Training Progress: 70%|███████ | 70/100 [1:08:23<39:27, 78.91s/it] (TaskRunner pid=2320396) step:71 - global_seqlen/min:19647 - global_seqlen/max:25487 - global_seqlen/minmax_diff:5840 - global_seqlen/balanced_min:22565 - global_seqlen/balanced_max:22577 - global_seqlen/mean:22573.375 - actor/entropy:0.23593924939632416 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151264250278473 - training/rollout_probs_diff_mean:0.0029731541872024536 - training/rollout_probs_diff_std:0.009116576053202152 - training/rollout_actor_probs_pearson_corr:0.9992273449897766 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.79296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9999300241470337 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9040335416793823 - rollout_corr/rollout_is_min:0.5837352871894836 - rollout_corr/rollout_is_std:0.028094211593270302 - rollout_corr/rollout_is_eff_sample_size:0.9992112187516006 - rollout_corr/rollout_is_seq_mean:0.9998959302902222 - rollout_corr/rollout_is_seq_std:0.0014871498569846153 - rollout_corr/rollout_is_seq_max:1.0080996751785278 - rollout_corr/rollout_is_seq_min:0.9946392774581909 - rollout_corr/rollout_is_seq_max_deviation:0.008099675178527832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2749516679905355) - rollout_corr/training_log_ppl:np.float64(0.22600234394485597) - rollout_corr/kl:np.float64(0.0004519108627860646) - rollout_corr/k3_kl:np.float64(0.00046148597355966103) - rollout_corr/rollout_ppl:np.float64(1.274333464447409) - rollout_corr/rollout_log_ppl:np.float64(0.22555043052852852) - rollout_corr/log_ppl_diff:np.float64(0.000451913416327443) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013323303937795572) - rollout_corr/log_ppl_diff_max:np.float64(0.00794076919555664) - rollout_corr/log_ppl_diff_min:np.float64(-0.004279732704162598) - rollout_corr/ppl_ratio:np.float64(1.0004535210318863) - rollout_corr/chi2_token:np.float64(0.0009360732510685921) - rollout_corr/chi2_seq:np.float64(0.4728632946498692) - actor/pg_loss:np.float64(0.002302838722243905) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025980098098443705) - actor/ppo_kl:np.float64(-5.338420736222815e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_w_mean:np.float64(32.36409391206689) - self_distillation/token_reweight_w_std:np.float64(696.1186733208597) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998391503468156) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045971906249178573) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.19924570620059967) - perf/mfu/actor:np.float64(0.053960876631776955) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.2897071838379) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003918695729225874 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003918695729225874 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:427.88671875 - response_length/max:1067.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.88671875 - response_length_non_aborted/max:1067.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011662766337394714 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5011277925223112) - timing_s/agent_loop/generate_sequences/max:np.float64(9.15895122103393) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.785454357996059) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.15895122103393) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:1067 - timing_s/gen:15.603826524689794 - timing_s/reward:0.05770381726324558 - timing_s/old_log_prob:3.1228378247469664 - timing_s/adv:0.6005111709237099 - timing_s/update_actor:21.588089572265744 - timing_s/step:41.09169472195208 - timing_s/stop_profile:2.989359200000763e-05 - timing_per_token_ms/gen:0.14244996325226444 - timing_per_token_ms/update_actor:0.11954398474012938 - timing_per_token_ms/adv:0.0033253289047589798 - perf/total_num_tokens:180587 - perf/time_per_step:41.09169472195208 - perf/throughput:549.341543412684 (TaskRunner pid=2320396) Training Progress: 71%|███████ | 71/100 [1:09:04<32:39, 67.57s/it] (TaskRunner pid=2320396) step:72 - global_seqlen/min:20651 - global_seqlen/max:27731 - global_seqlen/minmax_diff:7080 - global_seqlen/balanced_min:23755 - global_seqlen/balanced_max:23770 - global_seqlen/mean:23766.375 - actor/entropy:0.20936648547649384 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26683688163757324 - training/rollout_probs_diff_mean:0.002599770436063409 - training/rollout_probs_diff_std:0.008832783438265324 - training/rollout_actor_probs_pearson_corr:0.9991433620452881 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999456405639648 - rollout_corr/rollout_is_ratio_fraction_high:8.11089194030501e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.622178388061002e-05 - rollout_corr/rollout_is_max:2.0051612854003906 - rollout_corr/rollout_is_min:0.18509429693222046 - rollout_corr/rollout_is_std:0.02668849006295204 - rollout_corr/rollout_is_eff_sample_size:0.9992881124305862 - rollout_corr/rollout_is_seq_mean:0.9999668598175049 - rollout_corr/rollout_is_seq_std:0.0013168805744498968 - rollout_corr/rollout_is_seq_max:1.0059409141540527 - rollout_corr/rollout_is_seq_min:0.996266782283783 - rollout_corr/rollout_is_seq_max_deviation:0.005940914154052734 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2467334070242941) - rollout_corr/training_log_ppl:np.float64(0.1992745844036108) - rollout_corr/kl:np.float64(0.00042391312367939804) - rollout_corr/k3_kl:np.float64(0.00039017095170379434) - rollout_corr/rollout_ppl:np.float64(1.2461699997074902) - rollout_corr/rollout_log_ppl:np.float64(0.1988506696507102) - rollout_corr/log_ppl_diff:np.float64(0.00042391475290060043) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011057191295549273) - rollout_corr/log_ppl_diff_max:np.float64(0.0045042484998703) - rollout_corr/log_ppl_diff_min:np.float64(-0.004313677549362183) - rollout_corr/ppl_ratio:np.float64(1.0004249857738614) - rollout_corr/chi2_token:np.float64(0.0007191256154328585) - rollout_corr/chi2_seq:np.float64(0.29579704883508384) - actor/pg_loss:np.float64(0.001024734228849411) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015756982065795455) - actor/ppo_kl:np.float64(2.5644786637357697e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(8350.803719572723) - self_distillation/token_reweight_w_std:np.float64(178395.78476184432) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000230914214626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0306084052863298) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.11315388977527618) - perf/mfu/actor:np.float64(0.05624797651716909) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.9417266845703) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008435327559709549 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008435327559709549 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:481.60546875 - response_length/max:1367.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:481.60546875 - response_length_non_aborted/max:1367.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.057718276977539e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6083013713359833) - timing_s/agent_loop/generate_sequences/max:np.float64(10.97674540989101) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.481123702607874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.97674540989101) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1367 - timing_s/gen:17.8749917820096 - timing_s/reward:0.06322882324457169 - timing_s/old_log_prob:3.2556111719459295 - timing_s/adv:0.5855676289647818 - timing_s/update_actor:21.827434290200472 - timing_s/step:43.699696861207485 - timing_s/stop_profile:0.00011280179023742676 - timing_per_token_ms/gen:0.1449821299365696 - timing_per_token_ms/update_actor:0.11480208009320138 - timing_per_token_ms/adv:0.0030798114403478745 - perf/total_num_tokens:190131 - perf/time_per_step:43.699696861207485 - perf/throughput:543.8567474617329 (TaskRunner pid=2320396) Training Progress: 72%|███████▏ | 72/100 [1:09:48<28:11, 60.42s/it] (TaskRunner pid=2320396) step:73 - global_seqlen/min:17296 - global_seqlen/max:32968 - global_seqlen/minmax_diff:15672 - global_seqlen/balanced_min:25404 - global_seqlen/balanced_max:25442 - global_seqlen/mean:25428.5 - actor/entropy:0.254859983921051 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4140661358833313 - training/rollout_probs_diff_mean:0.0028677647933363914 - training/rollout_probs_diff_std:0.009029688313603401 - training/rollout_actor_probs_pearson_corr:0.9993501901626587 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.000032901763916 - rollout_corr/rollout_is_ratio_fraction_high:7.54398161006975e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.50879632201395e-05 - rollout_corr/rollout_is_max:2.0585145950317383 - rollout_corr/rollout_is_min:0.19157066941261292 - rollout_corr/rollout_is_std:0.028284510597586632 - rollout_corr/rollout_is_eff_sample_size:0.9992006259678756 - rollout_corr/rollout_is_seq_mean:0.9999836683273315 - rollout_corr/rollout_is_seq_std:0.0019239388639107347 - rollout_corr/rollout_is_seq_max:1.0077039003372192 - rollout_corr/rollout_is_seq_min:0.9925651550292969 - rollout_corr/rollout_is_seq_max_deviation:0.007703900337219238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3189547401852906) - rollout_corr/training_log_ppl:np.float64(0.2570040342397988) - rollout_corr/kl:np.float64(0.0005966623022146678) - rollout_corr/k3_kl:np.float64(0.0004898175338325927) - rollout_corr/rollout_ppl:np.float64(1.3181753954850137) - rollout_corr/rollout_log_ppl:np.float64(0.256407369626686) - rollout_corr/log_ppl_diff:np.float64(0.0005966646131128073) - rollout_corr/log_ppl_abs_diff:np.float64(0.00133663194719702) - rollout_corr/log_ppl_diff_max:np.float64(0.008886903524398804) - rollout_corr/log_ppl_diff_min:np.float64(-0.004705339670181274) - rollout_corr/ppl_ratio:np.float64(1.0005985633470118) - rollout_corr/chi2_token:np.float64(0.0007987350691109896) - rollout_corr/chi2_seq:np.float64(0.6767518501728773) - actor/pg_loss:np.float64(0.00291269866283983) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024224099979619496) - actor/ppo_kl:np.float64(0.00012902852559548705) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(7661.186577946413) - self_distillation/token_reweight_w_std:np.float64(198827.132493558) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001774257980287) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08138004870852455) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.22882431373000145) - perf/mfu/actor:np.float64(0.05922365864354845) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.86934280395508) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0030854884535074234 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0030854884535074234 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:517.796875 - response_length/max:1611.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:517.796875 - response_length_non_aborted/max:1611.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.911399006843567e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0221941787749529) - timing_s/agent_loop/generate_sequences/max:np.float64(12.645636655390263) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.6147617847746005) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.645636655390263) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:1611 - timing_s/gen:19.4119477532804 - timing_s/reward:0.06393673457205296 - timing_s/old_log_prob:3.1552902571856976 - timing_s/adv:0.6077743247151375 - timing_s/update_actor:22.31188929080963 - timing_s/step:45.63956543803215 - timing_s/stop_profile:0.00011852756142616272 - timing_per_token_ms/gen:0.14644337301427623 - timing_per_token_ms/update_actor:0.10967953915296631 - timing_per_token_ms/adv:0.0029876630784117105 - perf/total_num_tokens:203428 - perf/time_per_step:45.63956543803215 - perf/throughput:557.1591174444014 (TaskRunner pid=2320396) Training Progress: 73%|███████▎ | 73/100 [1:10:33<25:12, 56.01s/it] (TaskRunner pid=2320396) step:74 - global_seqlen/min:14990 - global_seqlen/max:28721 - global_seqlen/minmax_diff:13731 - global_seqlen/balanced_min:21228 - global_seqlen/balanced_max:21262 - global_seqlen/mean:21251.0 - actor/entropy:0.22410832345485687 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24179348349571228 - training/rollout_probs_diff_mean:0.0028740845154970884 - training/rollout_probs_diff_std:0.009236778132617474 - training/rollout_actor_probs_pearson_corr:0.999184250831604 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9999187588691711 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.471132216276601e-06 - rollout_corr/rollout_is_max:1.6295045614242554 - rollout_corr/rollout_is_min:0.3649487793445587 - rollout_corr/rollout_is_std:0.027934638783335686 - rollout_corr/rollout_is_eff_sample_size:0.9992200264057035 - rollout_corr/rollout_is_seq_mean:0.9999990463256836 - rollout_corr/rollout_is_seq_std:0.0019640212412923574 - rollout_corr/rollout_is_seq_max:1.006421446800232 - rollout_corr/rollout_is_seq_min:0.9917288422584534 - rollout_corr/rollout_is_seq_max_deviation:0.00827115774154663 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25452196970582) - rollout_corr/training_log_ppl:np.float64(0.21193978351948317) - rollout_corr/kl:np.float64(0.0006391342091154328) - rollout_corr/k3_kl:np.float64(0.0005525115491096244) - rollout_corr/rollout_ppl:np.float64(1.2536878855898976) - rollout_corr/rollout_log_ppl:np.float64(0.21130064985482022) - rollout_corr/log_ppl_diff:np.float64(0.0006391336646629497) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016215195209952071) - rollout_corr/log_ppl_diff_max:np.float64(0.01168176531791687) - rollout_corr/log_ppl_diff_min:np.float64(-0.00469110906124115) - rollout_corr/ppl_ratio:np.float64(1.0006418717093766) - rollout_corr/chi2_token:np.float64(0.0008710839319974184) - rollout_corr/chi2_seq:np.float64(0.41612958326004446) - actor/pg_loss:np.float64(0.0029548348393291235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002011250385294261) - actor/ppo_kl:np.float64(0.00016125536095801252) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(7153.264800689416) - self_distillation/token_reweight_w_std:np.float64(146011.8352878823) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995592245832086) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05097539708367549) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.20852339267730713) - perf/mfu/actor:np.float64(0.05059562385832781) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.92523574829102) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014293122105300426 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.014293122105300426 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:412.4375 - response_length/max:1764.0 - response_length/min:65.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.4375 - response_length_non_aborted/max:1764.0 - response_length_non_aborted/min:65.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.712654769420624e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6892342232167721) - timing_s/agent_loop/generate_sequences/max:np.float64(12.871262649074197) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.185971406070166) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.871262649074197) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1764 - timing_s/gen:20.324090790003538 - timing_s/reward:0.07555966824293137 - timing_s/old_log_prob:2.993329456076026 - timing_s/adv:0.5724999159574509 - timing_s/update_actor:21.911697022616863 - timing_s/step:45.928413731977344 - timing_s/stop_profile:0.00012182816863059998 - timing_per_token_ms/gen:0.19249214644267634 - timing_per_token_ms/update_actor:0.12888627019091375 - timing_per_token_ms/adv:0.0033674880944276203 - perf/total_num_tokens:170008 - perf/time_per_step:45.928413731977344 - perf/throughput:462.6983227422927 (TaskRunner pid=2320396) Training Progress: 74%|███████▍ | 74/100 [1:11:19<22:57, 53.00s/it] (TaskRunner pid=2320396) step:75 - global_seqlen/min:16466 - global_seqlen/max:23453 - global_seqlen/minmax_diff:6987 - global_seqlen/balanced_min:20955 - global_seqlen/balanced_max:20971 - global_seqlen/mean:20966.375 - actor/entropy:0.16500282287597656 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862035393714905 - training/rollout_probs_diff_mean:0.0026397460605949163 - training/rollout_probs_diff_std:0.009367442689836025 - training/rollout_actor_probs_pearson_corr:0.9988722801208496 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.8515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999569654464722 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.948394441977143e-05 - rollout_corr/rollout_is_max:1.7520591020584106 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.026879871264100075 - rollout_corr/rollout_is_eff_sample_size:0.9992779941651064 - rollout_corr/rollout_is_seq_mean:1.0000392198562622 - rollout_corr/rollout_is_seq_std:0.0016926469979807734 - rollout_corr/rollout_is_seq_max:1.0069493055343628 - rollout_corr/rollout_is_seq_min:0.9934012293815613 - rollout_corr/rollout_is_seq_max_deviation:0.006949305534362793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1966069960035384) - rollout_corr/training_log_ppl:np.float64(0.1667857915381319) - rollout_corr/kl:np.float64(0.0006250800824640912) - rollout_corr/k3_kl:np.float64(0.0005256048687840575) - rollout_corr/rollout_ppl:np.float64(1.195797340478748) - rollout_corr/rollout_log_ppl:np.float64(0.16616071118187392) - rollout_corr/log_ppl_diff:np.float64(0.0006250803562579677) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014622663875343278) - rollout_corr/log_ppl_diff_max:np.float64(0.022766180336475372) - rollout_corr/log_ppl_diff_min:np.float64(-0.005373738706111908) - rollout_corr/ppl_ratio:np.float64(1.000627742614597) - rollout_corr/chi2_token:np.float64(0.0006986984517425299) - rollout_corr/chi2_seq:np.float64(0.22746214317157865) - actor/pg_loss:np.float64(0.000660582329146564) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.942104457470123e-05) - actor/ppo_kl:np.float64(0.00026853492200018536) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_w_mean:np.float64(7268.040664529195) - self_distillation/token_reweight_w_std:np.float64(129162.24056490284) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002146703191102) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02054209852940403) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.12643406353890896) - perf/mfu/actor:np.float64(0.05026049795772646) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7984733581543) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003491861280053854 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003491861280053854 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.73046875 - response_length/max:1260.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.73046875 - response_length_non_aborted/max:1260.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002998523414134979 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2730316687375307) - timing_s/agent_loop/generate_sequences/max:np.float64(9.413474356755614) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.387236144888448) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.413474356755614) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:210 - timing_s/agent_loop/slowest/response_length:1260 - timing_s/gen:16.036076923832297 - timing_s/reward:0.08017385751008987 - timing_s/old_log_prob:3.003221595659852 - timing_s/adv:0.5643693823367357 - timing_s/update_actor:21.520387353375554 - timing_s/step:41.29089017212391 - timing_s/stop_profile:0.00011784397065639496 - timing_per_token_ms/gen:0.15829189418137243 - timing_per_token_ms/update_actor:0.12830298128178783 - timing_per_token_ms/adv:0.0033647291337721456 - perf/total_num_tokens:167731 - perf/time_per_step:41.29089017212391 - perf/throughput:507.7724145108092 (TaskRunner pid=2320396) Training Progress: 75%|███████▌ | 75/100 [1:12:01<20:37, 49.50s/it] (TaskRunner pid=2320396) step:76 - global_seqlen/min:21205 - global_seqlen/max:29111 - global_seqlen/minmax_diff:7906 - global_seqlen/balanced_min:24229 - global_seqlen/balanced_max:24247 - global_seqlen/mean:24241.625 - actor/entropy:0.23836244642734528 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6271570920944214 - training/rollout_probs_diff_mean:0.0028990982100367546 - training/rollout_probs_diff_std:0.009411140345036983 - training/rollout_actor_probs_pearson_corr:0.9991864562034607 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000035047531128 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.142990736814681e-06 - rollout_corr/rollout_is_max:1.7983005046844482 - rollout_corr/rollout_is_min:0.0746283084154129 - rollout_corr/rollout_is_std:0.0281493179500103 - rollout_corr/rollout_is_eff_sample_size:0.9992081242072904 - rollout_corr/rollout_is_seq_mean:1.0001025199890137 - rollout_corr/rollout_is_seq_std:0.0014056930085644126 - rollout_corr/rollout_is_seq_max:1.005411148071289 - rollout_corr/rollout_is_seq_min:0.9962686896324158 - rollout_corr/rollout_is_seq_max_deviation:0.0054111480712890625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2666503265500069) - rollout_corr/training_log_ppl:np.float64(0.22336477947828826) - rollout_corr/kl:np.float64(0.0004707449971448341) - rollout_corr/k3_kl:np.float64(0.00046742754221895666) - rollout_corr/rollout_ppl:np.float64(1.2659660731442273) - rollout_corr/rollout_log_ppl:np.float64(0.2228940339118708) - rollout_corr/log_ppl_diff:np.float64(0.0004707455664174631) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013132981694070622) - rollout_corr/log_ppl_diff_max:np.float64(0.00734102725982666) - rollout_corr/log_ppl_diff_min:np.float64(-0.004764363169670105) - rollout_corr/ppl_ratio:np.float64(1.0004723304882646) - rollout_corr/chi2_token:np.float64(0.0009256789926439524) - rollout_corr/chi2_seq:np.float64(0.6073360876180232) - actor/pg_loss:np.float64(0.0020736060105264187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002479879030943266) - actor/ppo_kl:np.float64(0.00013921088918422697) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(6886.147386563243) - self_distillation/token_reweight_w_std:np.float64(160346.53925711656) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005711058620363) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05048915263614617) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.15480851382017136) - perf/mfu/actor:np.float64(0.05722938996950414) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.76996278762817) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004289320670068264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004289320670068264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:479.70703125 - response_length/max:1260.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:479.70703125 - response_length_non_aborted/max:1260.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015643984079360962 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1107019297778606) - timing_s/agent_loop/generate_sequences/max:np.float64(10.500982820987701) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.217866806182428) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.500982820987701) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:1260 - timing_s/gen:17.480207895860076 - timing_s/reward:0.06496796384453773 - timing_s/old_log_prob:3.222787020727992 - timing_s/adv:0.5987678561359644 - timing_s/update_actor:21.95581777766347 - timing_s/step:43.41250845976174 - timing_s/stop_profile:0.0001221839338541031 - timing_per_token_ms/gen:0.14234117418557937 - timing_per_token_ms/update_actor:0.11321341792094934 - timing_per_token_ms/adv:0.0030874985491688594 - perf/total_num_tokens:193933 - perf/time_per_step:43.41250845976174 - perf/throughput:558.4018491460616 (TaskRunner pid=2320396) Training Progress: 76%|███████▌ | 76/100 [1:12:44<19:04, 47.70s/it] (TaskRunner pid=2320396) step:77 - global_seqlen/min:15619 - global_seqlen/max:27776 - global_seqlen/minmax_diff:12157 - global_seqlen/balanced_min:22253 - global_seqlen/balanced_max:22829 - global_seqlen/mean:22363.375 - actor/entropy:0.21923139691352844 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37990033626556396 - training/rollout_probs_diff_mean:0.002951013157144189 - training/rollout_probs_diff_std:0.009562992490828037 - training/rollout_actor_probs_pearson_corr:0.9991053342819214 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.999911367893219 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.550033216015436e-05 - rollout_corr/rollout_is_max:1.8682562112808228 - rollout_corr/rollout_is_min:0.2955019176006317 - rollout_corr/rollout_is_std:0.028421156108379364 - rollout_corr/rollout_is_eff_sample_size:0.9991928898070654 - rollout_corr/rollout_is_seq_mean:0.9998927712440491 - rollout_corr/rollout_is_seq_std:0.0016912176506593823 - rollout_corr/rollout_is_seq_max:1.005102276802063 - rollout_corr/rollout_is_seq_min:0.9950426816940308 - rollout_corr/rollout_is_seq_max_deviation:0.005102276802062988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2357686357572675) - rollout_corr/training_log_ppl:np.float64(0.20082158614241052) - rollout_corr/kl:np.float64(0.0005429030619339414) - rollout_corr/k3_kl:np.float64(0.00043220897507012523) - rollout_corr/rollout_ppl:np.float64(1.2350516240112484) - rollout_corr/rollout_log_ppl:np.float64(0.20027868168835994) - rollout_corr/log_ppl_diff:np.float64(0.0005429044540505856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013006862718611956) - rollout_corr/log_ppl_diff_max:np.float64(0.005264118313789368) - rollout_corr/log_ppl_diff_min:np.float64(-0.003429308533668518) - rollout_corr/ppl_ratio:np.float64(1.0005443205591291) - rollout_corr/chi2_token:np.float64(0.0006337540689855814) - rollout_corr/chi2_seq:np.float64(0.21167059405706823) - actor/pg_loss:np.float64(0.0019600478699430823) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016945441325333377) - actor/ppo_kl:np.float64(1.5344409057149733e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(5170.421208809828) - self_distillation/token_reweight_w_std:np.float64(99334.34433582227) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000085497740656) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03653434028092306) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.11112173949368298) - perf/mfu/actor:np.float64(0.05219431599433301) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.65237426757812) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015117594972252846 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015117594972252846 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.13671875 - response_length/max:2226.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.13671875 - response_length_non_aborted/max:2226.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011080503463745117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0522634759545326) - timing_s/agent_loop/generate_sequences/max:np.float64(15.5052381567657) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.6413939066260355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.5052381567657) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2226 - timing_s/gen:22.334846772253513 - timing_s/reward:0.06260121613740921 - timing_s/old_log_prob:3.061686584725976 - timing_s/adv:0.5881753247231245 - timing_s/update_actor:22.3996890950948 - timing_s/step:48.53496636450291 - timing_s/stop_profile:0.00013917125761508942 - timing_per_token_ms/gen:0.19822362345021977 - timing_per_token_ms/update_actor:0.12520297749721812 - timing_per_token_ms/adv:0.0032876037534759653 - perf/total_num_tokens:178907 - perf/time_per_step:48.53496636450291 - perf/throughput:460.7683217919346 (TaskRunner pid=2320396) Training Progress: 77%|███████▋ | 77/100 [1:13:33<18:23, 47.97s/it] (TaskRunner pid=2320396) step:78 - global_seqlen/min:16885 - global_seqlen/max:27989 - global_seqlen/minmax_diff:11104 - global_seqlen/balanced_min:22881 - global_seqlen/balanced_max:22910 - global_seqlen/mean:22902.875 - actor/entropy:0.19331404566764832 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35333073139190674 - training/rollout_probs_diff_mean:0.0028317312244325876 - training/rollout_probs_diff_std:0.009370970539748669 - training/rollout_actor_probs_pearson_corr:0.9989901781082153 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.84765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.0001708269119263 - rollout_corr/rollout_is_ratio_fraction_high:1.731077099975664e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.731077099975664e-05 - rollout_corr/rollout_is_max:2.9112932682037354 - rollout_corr/rollout_is_min:0.43616390228271484 - rollout_corr/rollout_is_std:0.028227554634213448 - rollout_corr/rollout_is_eff_sample_size:0.9992043155636828 - rollout_corr/rollout_is_seq_mean:1.0001126527786255 - rollout_corr/rollout_is_seq_std:0.001920209964737296 - rollout_corr/rollout_is_seq_max:1.0091001987457275 - rollout_corr/rollout_is_seq_min:0.9932641983032227 - rollout_corr/rollout_is_seq_max_deviation:0.009100198745727539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1990957544185221) - rollout_corr/training_log_ppl:np.float64(0.17567761093960144) - rollout_corr/kl:np.float64(0.0006657836468448863) - rollout_corr/k3_kl:np.float64(0.0007186990048140274) - rollout_corr/rollout_ppl:np.float64(1.1982321050018072) - rollout_corr/rollout_log_ppl:np.float64(0.17501182701380458) - rollout_corr/log_ppl_diff:np.float64(0.000665783925796859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016977235936792567) - rollout_corr/log_ppl_diff_max:np.float64(0.049715906381607056) - rollout_corr/log_ppl_diff_min:np.float64(-0.008080065250396729) - rollout_corr/ppl_ratio:np.float64(1.0006730714812875) - rollout_corr/chi2_token:np.float64(0.0011810176074504852) - rollout_corr/chi2_seq:np.float64(0.5433423959184438) - actor/pg_loss:np.float64(0.002108511747792363) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003408425704947149) - actor/ppo_kl:np.float64(0.00035315673385127866) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_w_mean:np.float64(6268.631206417689) - self_distillation/token_reweight_w_std:np.float64(116329.06896551698) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998350939713418) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0443158886191668) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.25990286469459534) - perf/mfu/actor:np.float64(0.05410870160542751) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.07009887695312) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028173280879855156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028173280879855156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.30859375 - response_length/max:1525.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.30859375 - response_length_non_aborted/max:1525.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499764621257782 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2836046293377876) - timing_s/agent_loop/generate_sequences/max:np.float64(11.574323669075966) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.801704231867916) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.574323669075966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:1525 - timing_s/gen:18.21794650144875 - timing_s/reward:0.06405134312808514 - timing_s/old_log_prob:3.1706241723150015 - timing_s/adv:0.5903529282659292 - timing_s/update_actor:22.47433365881443 - timing_s/step:44.60598493553698 - timing_s/stop_profile:0.00011158548295497894 - timing_per_token_ms/gen:0.1576833557056195 - timing_per_token_ms/update_actor:0.12266109417930299 - timing_per_token_ms/adv:0.003222045967296296 - perf/total_num_tokens:183223 - perf/time_per_step:44.60598493553698 - perf/throughput:513.4484763221446 (TaskRunner pid=2320396) Training Progress: 78%|███████▊ | 78/100 [1:14:17<17:13, 46.98s/it] (TaskRunner pid=2320396) step:79 - global_seqlen/min:16558 - global_seqlen/max:33044 - global_seqlen/minmax_diff:16486 - global_seqlen/balanced_min:24636 - global_seqlen/balanced_max:24664 - global_seqlen/mean:24657.875 - actor/entropy:0.19059540331363678 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016378581523895 - training/rollout_probs_diff_mean:0.0026734904386103153 - training/rollout_probs_diff_std:0.009236500598490238 - training/rollout_actor_probs_pearson_corr:0.9990987777709961 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.8203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.000053882598877 - rollout_corr/rollout_is_ratio_fraction_high:1.5713510947534814e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.856755473767407e-06 - rollout_corr/rollout_is_max:2.6087822914123535 - rollout_corr/rollout_is_min:0.4246424436569214 - rollout_corr/rollout_is_std:0.027441667392849922 - rollout_corr/rollout_is_eff_sample_size:0.9992476405980566 - rollout_corr/rollout_is_seq_mean:0.9999700784683228 - rollout_corr/rollout_is_seq_std:0.001676858519203961 - rollout_corr/rollout_is_seq_max:1.0092583894729614 - rollout_corr/rollout_is_seq_min:0.9936667680740356 - rollout_corr/rollout_is_seq_max_deviation:0.009258389472961426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2020120695233345) - rollout_corr/training_log_ppl:np.float64(0.17426249934942462) - rollout_corr/kl:np.float64(0.00046253042679245127) - rollout_corr/k3_kl:np.float64(0.0004328242781781455) - rollout_corr/rollout_ppl:np.float64(1.201419445220381) - rollout_corr/rollout_log_ppl:np.float64(0.1737999671822763) - rollout_corr/log_ppl_diff:np.float64(0.0004625321671483107) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013120870062266476) - rollout_corr/log_ppl_diff_max:np.float64(0.00897689163684845) - rollout_corr/log_ppl_diff_min:np.float64(-0.007229626178741455) - rollout_corr/ppl_ratio:np.float64(1.0004643604625016) - rollout_corr/chi2_token:np.float64(0.0007903354708105326) - rollout_corr/chi2_seq:np.float64(0.6352600273676217) - actor/pg_loss:np.float64(0.003112686681561172) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002658791520389059) - actor/ppo_kl:np.float64(2.6398326252063953e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_w_mean:np.float64(408.7717815169599) - self_distillation/token_reweight_w_std:np.float64(9884.658681469678) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990864836145192) - self_distillation/token_reweight_w_clip_frac:np.float64(0.054150840849615633) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.16461476869881153) - perf/mfu/actor:np.float64(0.05762919844007125) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.98615264892578) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006058540660887957 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006058540660887957 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:497.18359375 - response_length/max:1686.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:497.18359375 - response_length_non_aborted/max:1686.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.457976996898651e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4331816025078297) - timing_s/agent_loop/generate_sequences/max:np.float64(13.189415600150824) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.385525229481573) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.189415600150824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:1686 - timing_s/gen:19.859036806970835 - timing_s/reward:0.06584840640425682 - timing_s/old_log_prob:3.203118473291397 - timing_s/adv:0.6456434894353151 - timing_s/update_actor:22.172156451269984 - timing_s/step:46.03451617434621 - timing_s/stop_profile:0.00011187046766281128 - timing_per_token_ms/gen:0.1560275992659499 - timing_per_token_ms/update_actor:0.11239896205203198 - timing_per_token_ms/adv:0.003273008569449492 - perf/total_num_tokens:197263 - perf/time_per_step:46.03451617434621 - perf/throughput:535.6388433977105 (TaskRunner pid=2320396) Training Progress: 79%|███████▉ | 79/100 [1:15:03<16:20, 46.71s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. Using the formula for the electric field due to a point charge: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rearranging to solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the given values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) This value is closest to option **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2320396) step:80 - global_seqlen/min:17092 - global_seqlen/max:30502 - global_seqlen/minmax_diff:13410 - global_seqlen/balanced_min:23444 - global_seqlen/balanced_max:23520 - global_seqlen/mean:23455.875 - actor/entropy:0.1628781259059906 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851323008537292 - training/rollout_probs_diff_mean:0.00254165125079453 - training/rollout_probs_diff_std:0.009256764315068722 - training/rollout_actor_probs_pearson_corr:0.9989363551139832 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999262094497681 - rollout_corr/rollout_is_ratio_fraction_high:8.440314559265971e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.440314559265971e-06 - rollout_corr/rollout_is_max:2.55444073677063 - rollout_corr/rollout_is_min:0.05996687710285187 - rollout_corr/rollout_is_std:0.026260726153850555 - rollout_corr/rollout_is_eff_sample_size:0.9993107304708356 - rollout_corr/rollout_is_seq_mean:0.9999236464500427 - rollout_corr/rollout_is_seq_std:0.0016612117178738117 - rollout_corr/rollout_is_seq_max:1.0112076997756958 - rollout_corr/rollout_is_seq_min:0.9951014518737793 - rollout_corr/rollout_is_seq_max_deviation:0.0112076997756958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1665030946023762) - rollout_corr/training_log_ppl:np.float64(0.14808018820622237) - rollout_corr/kl:np.float64(0.0003466195788832671) - rollout_corr/k3_kl:np.float64(0.000394960443784953) - rollout_corr/rollout_ppl:np.float64(1.1661016917787492) - rollout_corr/rollout_log_ppl:np.float64(0.14773356900695944) - rollout_corr/log_ppl_diff:np.float64(0.00034661919926293194) - rollout_corr/log_ppl_abs_diff:np.float64(0.001185306638944894) - rollout_corr/log_ppl_diff_max:np.float64(0.0072307586669921875) - rollout_corr/log_ppl_diff_min:np.float64(-0.009507015347480774) - rollout_corr/ppl_ratio:np.float64(1.000348127214238) - rollout_corr/chi2_token:np.float64(0.0008883415721356869) - rollout_corr/chi2_seq:np.float64(0.43197265616618097) - actor/pg_loss:np.float64(0.001996967475861311) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033745979953891947) - actor/ppo_kl:np.float64(-0.00012091028241578705) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(23806.097967893817) - self_distillation/token_reweight_w_std:np.float64(548794.9111263388) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001761829480529) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04754826093267184) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.15800323337316513) - perf/mfu/actor:np.float64(0.05552541876185921) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.99964141845703) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.66640625) - val-aux/sciknoweval/reward/std@16:np.float64(0.15799927809691777) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7303875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12982374930438345) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.602) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13671534569253624) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6660875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.15808013005567143) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7812875000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09494219426925349) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.54905) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10555818229721874) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6775375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12453197346122731) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8188374999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06332935205697901) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.506175) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0748531273029532) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6804125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09154521211754381) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8425) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03640773357727453) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47509999999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04919681483385159) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06299643554923988) - val-aux/sciknoweval/score/mean@16:np.float64(0.66640625) - val-aux/sciknoweval/score/std@16:np.float64(0.15799927809691777) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7303875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12982374930438345) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.602) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13671534569253624) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6660875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.15808013005567143) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7812875000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09494219426925349) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.54905) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10555818229721874) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6775375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12453197346122731) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8188374999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06332935205697901) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.506175) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0748531273029532) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6804125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09154521211754381) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8425) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03640773357727453) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47509999999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04919681483385159) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06299643554923988) - val-core/sciknoweval/acc/mean@16:np.float64(0.66640625) - val-aux/sciknoweval/acc/std@16:np.float64(0.15799927809691777) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7303875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12982374930438345) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.602) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13671534569253624) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6660875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.15808013005567143) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7812875000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09494219426925349) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.54905) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10555818229721874) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6775375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12453197346122731) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8188374999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06332935205697901) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.506175) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0748531273029532) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6804125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09154521211754381) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8425) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03640773357727453) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47509999999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04919681483385159) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6779125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06299643554923988) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9953125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.00881967330726803) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.004383097536411576) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9921) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.01028015103724269) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.995725) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.008610222519927225) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.001243734296383275) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.987675) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01048649118826383) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9985875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004890128461522432) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9830249999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.008839286092192807) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0023919993524231077) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9794750000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006665041979425554) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008666092529892921 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008666092529892921 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:462.80859375 - response_length/max:1515.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:462.80859375 - response_length_non_aborted/max:1515.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010422617197036743 - timing_s/agent_loop/generate_sequences/min:np.float64(1.178299294784665) - timing_s/agent_loop/generate_sequences/max:np.float64(11.539570804685354) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.233297567705449) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.539570804685354) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1515 - timing_s/gen:18.76097808778286 - timing_s/reward:0.06370272673666477 - timing_s/old_log_prob:3.0534738190472126 - timing_s/adv:0.6413047388195992 - timing_s/update_actor:22.003460492938757 - timing_s/step:44.61081040836871 - timing_s/testing:101.52909150719643 - timing_s/save_checkpoint:13.23071464523673 - timing_s/stop_profile:0.00013984739780426025 - timing_per_token_ms/gen:0.15834855196096237 - timing_per_token_ms/update_actor:0.11725985756734057 - timing_per_token_ms/adv:0.003417612532145993 - perf/total_num_tokens:187647 - perf/time_per_step:44.61081040836871 - perf/throughput:525.7890360045965 (TaskRunner pid=2320396) Training Progress: 80%|████████ | 80/100 [1:17:43<26:50, 80.53s/it] (TaskRunner pid=2320396) step:81 - global_seqlen/min:18108 - global_seqlen/max:30762 - global_seqlen/minmax_diff:12654 - global_seqlen/balanced_min:22819 - global_seqlen/balanced_max:22852 - global_seqlen/mean:22838.625 - actor/entropy:0.20225822925567627 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3244411051273346 - training/rollout_probs_diff_mean:0.002944577718153596 - training/rollout_probs_diff_std:0.009473892860114574 - training/rollout_actor_probs_pearson_corr:0.9991171360015869 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0000029802322388 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.597196028858889e-06 - rollout_corr/rollout_is_max:1.6389256715774536 - rollout_corr/rollout_is_min:0.3698889911174774 - rollout_corr/rollout_is_std:0.028404373675584793 - rollout_corr/rollout_is_eff_sample_size:0.9991939609606479 - rollout_corr/rollout_is_seq_mean:0.9999433755874634 - rollout_corr/rollout_is_seq_std:0.0018434900557622313 - rollout_corr/rollout_is_seq_max:1.0091862678527832 - rollout_corr/rollout_is_seq_min:0.9904931783676147 - rollout_corr/rollout_is_seq_max_deviation:0.009506821632385254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2227952620014548) - rollout_corr/training_log_ppl:np.float64(0.19357780196150998) - rollout_corr/kl:np.float64(0.0005325028031997192) - rollout_corr/k3_kl:np.float64(0.0005149519024598703) - rollout_corr/rollout_ppl:np.float64(1.2221037549898028) - rollout_corr/rollout_log_ppl:np.float64(0.19304529835790163) - rollout_corr/log_ppl_diff:np.float64(0.0005325036036083475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014249040541471913) - rollout_corr/log_ppl_diff_max:np.float64(0.010488063097000122) - rollout_corr/log_ppl_diff_min:np.float64(-0.008635297417640686) - rollout_corr/ppl_ratio:np.float64(1.0005347745027393) - rollout_corr/chi2_token:np.float64(0.0009839723352342844) - rollout_corr/chi2_seq:np.float64(0.3460533053148538) - actor/pg_loss:np.float64(0.0018761196406558156) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002735087759901944) - actor/ppo_kl:np.float64(2.139973337733636e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(36.513663759920746) - self_distillation/token_reweight_w_std:np.float64(801.7813025001087) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010359066072851) - self_distillation/token_reweight_w_clip_frac:np.float64(0.035162036278052256) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.14915302395820618) - perf/mfu/actor:np.float64(0.05383724667218048) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.4105339050293) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00048574156244285405 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00048574156244285405 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:454.36328125 - response_length/max:1584.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:454.36328125 - response_length_non_aborted/max:1584.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.142312824726105e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1165193412452936) - timing_s/agent_loop/generate_sequences/max:np.float64(12.071419509127736) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.783302926065517) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.071419509127736) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:1584 - timing_s/gen:18.30917154625058 - timing_s/reward:0.059204936027526855 - timing_s/old_log_prob:3.1515744253993034 - timing_s/adv:0.6499936617910862 - timing_s/update_actor:22.073494616895914 - timing_s/step:44.343954378739 - timing_s/stop_profile:0.0001219436526298523 - timing_per_token_ms/gen:0.15740752896180765 - timing_per_token_ms/update_actor:0.12081230052649795 - timing_per_token_ms/adv:0.0035575349971325235 - perf/total_num_tokens:182709 - perf/time_per_step:44.343954378739 - perf/throughput:515.0335670323107 (TaskRunner pid=2320396) Training Progress: 81%|████████ | 81/100 [1:18:27<22:04, 69.69s/it] (TaskRunner pid=2320396) step:82 - global_seqlen/min:18946 - global_seqlen/max:27759 - global_seqlen/minmax_diff:8813 - global_seqlen/balanced_min:24293 - global_seqlen/balanced_max:24305 - global_seqlen/mean:24300.75 - actor/entropy:0.1955031305551529 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19416940212249756 - training/rollout_probs_diff_mean:0.0027140500023961067 - training/rollout_probs_diff_std:0.009143715724349022 - training/rollout_actor_probs_pearson_corr:0.9990493059158325 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.00007963180542 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.615221844986081e-05 - rollout_corr/rollout_is_max:1.8455089330673218 - rollout_corr/rollout_is_min:0.38921883702278137 - rollout_corr/rollout_is_std:0.02782774716615677 - rollout_corr/rollout_is_eff_sample_size:0.9992262156609376 - rollout_corr/rollout_is_seq_mean:1.0001100301742554 - rollout_corr/rollout_is_seq_std:0.0014895005151629448 - rollout_corr/rollout_is_seq_max:1.0084476470947266 - rollout_corr/rollout_is_seq_min:0.9956014156341553 - rollout_corr/rollout_is_seq_max_deviation:0.008447647094726562 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.214292073622346) - rollout_corr/training_log_ppl:np.float64(0.18476674790144898) - rollout_corr/kl:np.float64(0.0003596869540132275) - rollout_corr/k3_kl:np.float64(0.00045131250168140014) - rollout_corr/rollout_ppl:np.float64(1.2138330452144146) - rollout_corr/rollout_log_ppl:np.float64(0.18440705924876966) - rollout_corr/log_ppl_diff:np.float64(0.00035968865267932415) - rollout_corr/log_ppl_abs_diff:np.float64(0.001154881319962442) - rollout_corr/log_ppl_diff_max:np.float64(0.0048035383224487305) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042536258697509766) - rollout_corr/ppl_ratio:np.float64(1.0003608274273574) - rollout_corr/chi2_token:np.float64(0.0010943859815597534) - rollout_corr/chi2_seq:np.float64(0.6024468285031617) - actor/pg_loss:np.float64(0.0022809606743976474) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001371511980323703) - actor/ppo_kl:np.float64(7.002319382820588e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(3974.396525104763) - self_distillation/token_reweight_w_std:np.float64(79235.03549689136) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997991595882922) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03545600798679516) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.140237957239151) - perf/mfu/actor:np.float64(0.056121153281023836) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.15105438232422) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014725775457918644 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014725775457918644 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:483.6796875 - response_length/max:1464.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:483.6796875 - response_length_non_aborted/max:1464.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001256559044122696 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1039730682969093) - timing_s/agent_loop/generate_sequences/max:np.float64(11.90431728027761) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.50781928942888) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.90431728027761) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1464 - timing_s/gen:18.580683009698987 - timing_s/reward:0.0608761440962553 - timing_s/old_log_prob:3.0198480412364006 - timing_s/adv:0.5776665359735489 - timing_s/update_actor:22.33436026982963 - timing_s/step:44.61843103542924 - timing_s/stop_profile:0.00011967122554779053 - timing_per_token_ms/gen:0.15005962599294945 - timing_per_token_ms/update_actor:0.114885138677971 - timing_per_token_ms/adv:0.002971443967642711 - perf/total_num_tokens:194406 - perf/time_per_step:44.61843103542924 - perf/throughput:544.6347940989679 (TaskRunner pid=2320396) Training Progress: 82%|████████▏ | 82/100 [1:19:12<18:39, 62.18s/it] (TaskRunner pid=2320396) step:83 - global_seqlen/min:17814 - global_seqlen/max:27538 - global_seqlen/minmax_diff:9724 - global_seqlen/balanced_min:22306 - global_seqlen/balanced_max:22318 - global_seqlen/mean:22313.375 - actor/entropy:0.21886374056339264 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016381561756134 - training/rollout_probs_diff_mean:0.003045517485588789 - training/rollout_probs_diff_std:0.00972580537199974 - training/rollout_actor_probs_pearson_corr:0.9990494847297668 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.77734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:1.0000078678131104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.788648271234706e-05 - rollout_corr/rollout_is_max:1.8235840797424316 - rollout_corr/rollout_is_min:0.38655397295951843 - rollout_corr/rollout_is_std:0.029688945040106773 - rollout_corr/rollout_is_eff_sample_size:0.9991191047550716 - rollout_corr/rollout_is_seq_mean:0.9999637007713318 - rollout_corr/rollout_is_seq_std:0.0019075077725574374 - rollout_corr/rollout_is_seq_max:1.0059213638305664 - rollout_corr/rollout_is_seq_min:0.9916067719459534 - rollout_corr/rollout_is_seq_max_deviation:0.00839322805404663 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2587532331235707) - rollout_corr/training_log_ppl:np.float64(0.215623820280598) - rollout_corr/kl:np.float64(0.00042866195441604305) - rollout_corr/k3_kl:np.float64(0.0005887536341475652) - rollout_corr/rollout_ppl:np.float64(1.2582748807035387) - rollout_corr/rollout_log_ppl:np.float64(0.21519515893305652) - rollout_corr/log_ppl_diff:np.float64(0.00042866134754149243) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015262856832123362) - rollout_corr/log_ppl_diff_max:np.float64(0.010632097721099854) - rollout_corr/log_ppl_diff_min:np.float64(-0.0100230872631073) - rollout_corr/ppl_ratio:np.float64(1.000431136926636) - rollout_corr/chi2_token:np.float64(0.0015211885329335928) - rollout_corr/chi2_seq:np.float64(1.5993264480493963) - actor/pg_loss:np.float64(0.003375511965714395) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044200276761330315) - actor/ppo_kl:np.float64(-9.806516564747625e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_w_mean:np.float64(13916.900597623782) - self_distillation/token_reweight_w_std:np.float64(302939.21583341405) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994024811312556) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06193480538786389) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2252168245613575) - perf/mfu/actor:np.float64(0.05314491856218738) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.17446899414062) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007381784729659557 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007381784729659557 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.23046875 - response_length/max:1116.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.23046875 - response_length_non_aborted/max:1116.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001424010843038559 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7883939687162638) - timing_s/agent_loop/generate_sequences/max:np.float64(9.522131288424134) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.651366425387096) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.522131288424134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1116 - timing_s/gen:16.62836728617549 - timing_s/reward:0.06175902858376503 - timing_s/old_log_prob:3.2153992895036936 - timing_s/adv:0.6615845076739788 - timing_s/update_actor:21.87908287346363 - timing_s/step:42.53540291264653 - timing_s/stop_profile:0.00012497231364250183 - timing_per_token_ms/gen:0.15456889621743547 - timing_per_token_ms/update_actor:0.12256708629613197 - timing_per_token_ms/adv:0.0037062104437023693 - perf/total_num_tokens:178507 - perf/time_per_step:42.53540291264653 - perf/throughput:524.5836049989746 (TaskRunner pid=2320396) Training Progress: 83%|████████▎ | 83/100 [1:19:54<15:57, 56.30s/it] (TaskRunner pid=2320396) step:84 - global_seqlen/min:19307 - global_seqlen/max:29887 - global_seqlen/minmax_diff:10580 - global_seqlen/balanced_min:23081 - global_seqlen/balanced_max:23100 - global_seqlen/mean:23097.0 - actor/entropy:0.2354067713022232 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35205382108688354 - training/rollout_probs_diff_mean:0.003032247070223093 - training/rollout_probs_diff_std:0.009490408934652805 - training/rollout_actor_probs_pearson_corr:0.9991567730903625 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000667572021484 - rollout_corr/rollout_is_ratio_fraction_high:8.543503099645022e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.543503099645022e-06 - rollout_corr/rollout_is_max:2.8205838203430176 - rollout_corr/rollout_is_min:0.21296259760856628 - rollout_corr/rollout_is_std:0.028636353090405464 - rollout_corr/rollout_is_eff_sample_size:0.9991808692410266 - rollout_corr/rollout_is_seq_mean:1.000049352645874 - rollout_corr/rollout_is_seq_std:0.0015556849539279938 - rollout_corr/rollout_is_seq_max:1.00594961643219 - rollout_corr/rollout_is_seq_min:0.9951849579811096 - rollout_corr/rollout_is_seq_max_deviation:0.005949616432189941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2848193626850843) - rollout_corr/training_log_ppl:np.float64(0.22980090814235155) - rollout_corr/kl:np.float64(0.0004010426303544534) - rollout_corr/k3_kl:np.float64(0.0005172141459865998) - rollout_corr/rollout_ppl:np.float64(1.284251406788826) - rollout_corr/rollout_log_ppl:np.float64(0.22939986467099516) - rollout_corr/log_ppl_diff:np.float64(0.0004010434713563882) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013402195836533792) - rollout_corr/log_ppl_diff_max:np.float64(0.0082988440990448) - rollout_corr/log_ppl_diff_min:np.float64(-0.015117630362510681) - rollout_corr/ppl_ratio:np.float64(1.0004031020216644) - rollout_corr/chi2_token:np.float64(0.0013102053198963404) - rollout_corr/chi2_seq:np.float64(0.600376995280385) - actor/pg_loss:np.float64(0.002821797039359808) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017985764498007484) - actor/ppo_kl:np.float64(1.2684636487847456e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(3089.401668926468) - self_distillation/token_reweight_w_std:np.float64(78661.03726228699) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996353969909251) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05245249951258302) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.16984390653669834) - perf/mfu/actor:np.float64(0.05433426907698578) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.13277435302734) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003203813685104251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003203813685104251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.21875 - response_length/max:1337.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.21875 - response_length_non_aborted/max:1337.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015847571194171906 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1861316971480846) - timing_s/agent_loop/generate_sequences/max:np.float64(10.804910905659199) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.034344409927144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.804910905659199) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:1337 - timing_s/gen:17.917205162346363 - timing_s/reward:0.05784008279442787 - timing_s/old_log_prob:3.1344593465328217 - timing_s/adv:0.6041016634553671 - timing_s/update_actor:22.013284035027027 - timing_s/step:43.842269690707326 - timing_s/stop_profile:0.00012374669313430786 - timing_per_token_ms/gen:0.15307570537169676 - timing_per_token_ms/update_actor:0.11913497442864347 - timing_per_token_ms/adv:0.003269372989215954 - perf/total_num_tokens:184776 - perf/time_per_step:43.842269690707326 - perf/throughput:526.8203531190716 (TaskRunner pid=2320396) Training Progress: 84%|████████▍ | 84/100 [1:20:38<14:01, 52.58s/it] (TaskRunner pid=2320396) step:85 - global_seqlen/min:15110 - global_seqlen/max:28870 - global_seqlen/minmax_diff:13760 - global_seqlen/balanced_min:22104 - global_seqlen/balanced_max:22120 - global_seqlen/mean:22112.25 - actor/entropy:0.21829085052013397 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25521230697631836 - training/rollout_probs_diff_mean:0.002716940827667713 - training/rollout_probs_diff_std:0.009013369679450989 - training/rollout_actor_probs_pearson_corr:0.9992413520812988 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8207971152151003e-05 - rollout_corr/rollout_is_max:1.9953315258026123 - rollout_corr/rollout_is_min:0.2880904972553253 - rollout_corr/rollout_is_std:0.0274242851883173 - rollout_corr/rollout_is_eff_sample_size:0.9992487118690302 - rollout_corr/rollout_is_seq_mean:1.0001178979873657 - rollout_corr/rollout_is_seq_std:0.0018713462632149458 - rollout_corr/rollout_is_seq_max:1.0108190774917603 - rollout_corr/rollout_is_seq_min:0.9933320879936218 - rollout_corr/rollout_is_seq_max_deviation:0.010819077491760254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2405682974494994) - rollout_corr/training_log_ppl:np.float64(0.2008585182193201) - rollout_corr/kl:np.float64(0.00034136320786615215) - rollout_corr/k3_kl:np.float64(0.00045656332018495505) - rollout_corr/rollout_ppl:np.float64(1.240147914737463) - rollout_corr/rollout_log_ppl:np.float64(0.2005171545824851) - rollout_corr/log_ppl_diff:np.float64(0.0003413636368350126) - rollout_corr/log_ppl_abs_diff:np.float64(0.001429379561159294) - rollout_corr/log_ppl_diff_max:np.float64(0.006210342049598694) - rollout_corr/log_ppl_diff_min:np.float64(-0.009608462452888489) - rollout_corr/ppl_ratio:np.float64(1.000343416351825) - rollout_corr/chi2_token:np.float64(0.0011162424925714731) - rollout_corr/chi2_seq:np.float64(0.7462084121070802) - actor/pg_loss:np.float64(0.002974406350404024) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002645291015141993) - actor/ppo_kl:np.float64(5.648398862057036e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(819.4280955540016) - self_distillation/token_reweight_w_std:np.float64(19350.410570348613) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996866395231336) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0477723392250482) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.1905206125229597) - perf/mfu/actor:np.float64(0.051957974999849506) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.11687469482422) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004917290061712265 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004917290061712265 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:429.0703125 - response_length/max:1578.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:429.0703125 - response_length_non_aborted/max:1578.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001225452870130539 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7567968033254147) - timing_s/agent_loop/generate_sequences/max:np.float64(12.397239990532398) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.498732063300849) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.397239990532398) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:363 - timing_s/agent_loop/slowest/response_length:1578 - timing_s/gen:19.835766257718205 - timing_s/reward:0.0627602580934763 - timing_s/old_log_prob:3.027434177696705 - timing_s/adv:0.630172599107027 - timing_s/update_actor:22.07317166402936 - timing_s/step:45.71889763139188 - timing_s/stop_profile:0.00011015869677066803 - timing_per_token_ms/gen:0.18058453285371903 - timing_per_token_ms/update_actor:0.12477909113743152 - timing_per_token_ms/adv:0.003562350049785905 - perf/total_num_tokens:176898 - perf/time_per_step:45.71889763139188 - perf/throughput:483.65667471424564 (TaskRunner pid=2320396) Training Progress: 85%|████████▌ | 85/100 [1:21:24<12:38, 50.54s/it] (TaskRunner pid=2320396) step:86 - global_seqlen/min:18259 - global_seqlen/max:24454 - global_seqlen/minmax_diff:6195 - global_seqlen/balanced_min:22310 - global_seqlen/balanced_max:22317 - global_seqlen/mean:22315.25 - actor/entropy:0.19693319499492645 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9269568920135498 - training/rollout_probs_diff_mean:0.002871491713449359 - training/rollout_probs_diff_std:0.010464465245604515 - training/rollout_actor_probs_pearson_corr:0.99885493516922 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0000848770141602 - rollout_corr/rollout_is_ratio_fraction_high:1.8361763068242e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.5085289204725996e-05 - rollout_corr/rollout_is_max:4.428065299987793 - rollout_corr/rollout_is_min:0.00090052280575037 - rollout_corr/rollout_is_std:0.0283602736890316 - rollout_corr/rollout_is_eff_sample_size:0.999196341310165 - rollout_corr/rollout_is_seq_mean:1.0001143217086792 - rollout_corr/rollout_is_seq_std:0.001787590328603983 - rollout_corr/rollout_is_seq_max:1.0075844526290894 - rollout_corr/rollout_is_seq_min:0.9934707283973694 - rollout_corr/rollout_is_seq_max_deviation:0.0075844526290893555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2275358876213431) - rollout_corr/training_log_ppl:np.float64(0.19244293621886754) - rollout_corr/kl:np.float64(0.0004759999551815497) - rollout_corr/k3_kl:np.float64(0.0006213987889509553) - rollout_corr/rollout_ppl:np.float64(1.22697224188596) - rollout_corr/rollout_log_ppl:np.float64(0.19196693430421874) - rollout_corr/log_ppl_diff:np.float64(0.0004760019146488048) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014411285155802034) - rollout_corr/log_ppl_diff_max:np.float64(0.009107738733291626) - rollout_corr/log_ppl_diff_min:np.float64(-0.007479369640350342) - rollout_corr/ppl_ratio:np.float64(1.0004782637115568) - rollout_corr/chi2_token:np.float64(0.001509897643700242) - rollout_corr/chi2_seq:np.float64(0.7807369504589587) - actor/pg_loss:np.float64(0.003120149136520922) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000514148305001072) - actor/ppo_kl:np.float64(1.555324082502807e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(28969.59762649727) - self_distillation/token_reweight_w_std:np.float64(541568.6326414472) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999872143380344) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05786374416493345) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.21097707375884056) - perf/mfu/actor:np.float64(0.05366693062582211) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.08077239990234) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0003144452057313174 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0003144452057313174 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:425.4765625 - response_length/max:1046.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.4765625 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001625940203666687 - timing_s/agent_loop/generate_sequences/min:np.float64(1.250269953161478) - timing_s/agent_loop/generate_sequences/max:np.float64(9.116639155894518) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7879958484263625) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.116639155894518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:15.873301655054092 - timing_s/reward:0.06210778281092644 - timing_s/old_log_prob:2.9981462005525827 - timing_s/adv:0.5681248959153891 - timing_s/update_actor:21.514174001291394 - timing_s/step:41.10377608239651 - timing_s/stop_profile:2.7652829885482788e-05 - timing_per_token_ms/gen:0.14573090518953097 - timing_per_token_ms/update_actor:0.12051273233154118 - timing_per_token_ms/adv:0.0031823802999932167 - perf/total_num_tokens:178522 - perf/time_per_step:41.10377608239651 - perf/throughput:542.9002424319098 (TaskRunner pid=2320396) Training Progress: 86%|████████▌ | 86/100 [1:22:05<11:07, 47.71s/it] (TaskRunner pid=2320396) step:87 - global_seqlen/min:20010 - global_seqlen/max:25380 - global_seqlen/minmax_diff:5370 - global_seqlen/balanced_min:22327 - global_seqlen/balanced_max:22346 - global_seqlen/mean:22340.875 - actor/entropy:0.2229112833738327 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28752514719963074 - training/rollout_probs_diff_mean:0.00292000244371593 - training/rollout_probs_diff_std:0.009206824004650116 - training/rollout_actor_probs_pearson_corr:0.9992163777351379 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.99995356798172 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.715296770678833e-05 - rollout_corr/rollout_is_max:1.7399653196334839 - rollout_corr/rollout_is_min:0.386408269405365 - rollout_corr/rollout_is_std:0.027554381638765335 - rollout_corr/rollout_is_eff_sample_size:0.999241213020451 - rollout_corr/rollout_is_seq_mean:0.9999908208847046 - rollout_corr/rollout_is_seq_std:0.0016598277725279331 - rollout_corr/rollout_is_seq_max:1.0081655979156494 - rollout_corr/rollout_is_seq_min:0.9906253814697266 - rollout_corr/rollout_is_seq_max_deviation:0.009374618530273438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2702338360249996) - rollout_corr/training_log_ppl:np.float64(0.22234329585626256) - rollout_corr/kl:np.float64(0.0005667585143394049) - rollout_corr/k3_kl:np.float64(0.0004980311200313281) - rollout_corr/rollout_ppl:np.float64(1.2695111935026944) - rollout_corr/rollout_log_ppl:np.float64(0.22177653654216556) - rollout_corr/log_ppl_diff:np.float64(0.0005667593140969984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014052593251108192) - rollout_corr/log_ppl_diff_max:np.float64(0.012764573097229004) - rollout_corr/log_ppl_diff_min:np.float64(-0.004403352737426758) - rollout_corr/ppl_ratio:np.float64(1.000568661140278) - rollout_corr/chi2_token:np.float64(0.000875755213201046) - rollout_corr/chi2_seq:np.float64(0.24442025600001216) - actor/pg_loss:np.float64(0.0031820654403418303) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029767151454507257) - actor/ppo_kl:np.float64(0.00015311227060976762) - actor/pg_clipfrac_lower:np.float64(4.398930286697578e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(1033.2029310131911) - self_distillation/token_reweight_w_std:np.float64(21538.62358322338) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992069352883846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06586394051555544) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.21494160592556) - perf/mfu/actor:np.float64(0.05288507261784649) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.0767822265625) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029490168672055006 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029490168672055006 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:420.55859375 - response_length/max:1262.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.55859375 - response_length_non_aborted/max:1262.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.940774619579315e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.18855987675488) - timing_s/agent_loop/generate_sequences/max:np.float64(9.835301402956247) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.683391153987031) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.835301402956247) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:256 - timing_s/agent_loop/slowest/response_length:1262 - timing_s/gen:16.892137264832854 - timing_s/reward:0.06300836428999901 - timing_s/old_log_prob:2.9979468770325184 - timing_s/adv:0.5653424803167582 - timing_s/update_actor:22.004743106663227 - timing_s/step:42.61258958093822 - timing_s/stop_profile:0.00011842325329780579 - timing_per_token_ms/gen:0.15689825905680554 - timing_per_token_ms/update_actor:0.12311929986327319 - timing_per_token_ms/adv:0.0031631621429149383 - perf/total_num_tokens:178727 - perf/time_per_step:42.61258958093822 - perf/throughput:524.2787453122465 (TaskRunner pid=2320396) Training Progress: 87%|████████▋ | 87/100 [1:22:48<10:00, 46.20s/it] (TaskRunner pid=2320396) step:88 - global_seqlen/min:18099 - global_seqlen/max:28822 - global_seqlen/minmax_diff:10723 - global_seqlen/balanced_min:23419 - global_seqlen/balanced_max:23442 - global_seqlen/mean:23436.25 - actor/entropy:0.21247567236423492 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29311496019363403 - training/rollout_probs_diff_mean:0.0028727473691105843 - training/rollout_probs_diff_std:0.00918804481625557 - training/rollout_actor_probs_pearson_corr:0.9991670250892639 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9998396039009094 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.663644434302114e-05 - rollout_corr/rollout_is_max:1.599510669708252 - rollout_corr/rollout_is_min:0.4783361852169037 - rollout_corr/rollout_is_std:0.027528412640094757 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:0.9999282360076904 - rollout_corr/rollout_is_seq_std:0.0018976305145770311 - rollout_corr/rollout_is_seq_max:1.0152655839920044 - rollout_corr/rollout_is_seq_min:0.9939031600952148 - rollout_corr/rollout_is_seq_max_deviation:0.015265583992004395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2466978626325727) - rollout_corr/training_log_ppl:np.float64(0.2076665610074997) - rollout_corr/kl:np.float64(0.00044316515954800906) - rollout_corr/k3_kl:np.float64(0.00045984440038182584) - rollout_corr/rollout_ppl:np.float64(1.2460913262329996) - rollout_corr/rollout_log_ppl:np.float64(0.20722339424537495) - rollout_corr/log_ppl_diff:np.float64(0.00044316676212474704) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013816133432555944) - rollout_corr/log_ppl_diff_max:np.float64(0.006623566150665283) - rollout_corr/log_ppl_diff_min:np.float64(-0.01356533169746399) - rollout_corr/ppl_ratio:np.float64(1.0004452236462384) - rollout_corr/chi2_token:np.float64(0.0009807520546019077) - rollout_corr/chi2_seq:np.float64(2.0590124474838376) - actor/pg_loss:np.float64(0.003491878043860197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002180453784603742) - actor/ppo_kl:np.float64(-3.431808095299971e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(4024.9873863377143) - self_distillation/token_reweight_w_std:np.float64(95798.2680301803) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008977930992842) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06488000725221355) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.18007886037230492) - perf/mfu/actor:np.float64(0.0547133357028863) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.0287628173828) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004015621729195118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004015621729195118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:469.6015625 - response_length/max:1509.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:469.6015625 - response_length_non_aborted/max:1509.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010225735604763031 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9620911218225956) - timing_s/agent_loop/generate_sequences/max:np.float64(11.938467379659414) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.014028608202352) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.938467379659414) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1509 - timing_s/gen:18.666382912546396 - timing_s/reward:0.06460434012115002 - timing_s/old_log_prob:3.040660046041012 - timing_s/adv:0.6120849438011646 - timing_s/update_actor:22.133079420775175 - timing_s/step:44.60495771467686 - timing_s/stop_profile:0.00011262297630310059 - timing_per_token_ms/gen:0.15527111507882677 - timing_per_token_ms/update_actor:0.11804938621139888 - timing_per_token_ms/adv:0.0032646271470540542 - perf/total_num_tokens:187490 - perf/time_per_step:44.60495771467686 - perf/throughput:525.4180521796238 (TaskRunner pid=2320396) Training Progress: 88%|████████▊ | 88/100 [1:23:33<09:08, 45.73s/it] (TaskRunner pid=2320396) step:89 - global_seqlen/min:14328 - global_seqlen/max:25446 - global_seqlen/minmax_diff:11118 - global_seqlen/balanced_min:20881 - global_seqlen/balanced_max:20887 - global_seqlen/mean:20883.875 - actor/entropy:0.2559927701950073 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2545143961906433 - training/rollout_probs_diff_mean:0.0030213401187211275 - training/rollout_probs_diff_std:0.009110929444432259 - training/rollout_actor_probs_pearson_corr:0.9992585778236389 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:1.0 - self_distillation/correct_sample_fraction:0.8671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999969601631165 - rollout_corr/rollout_is_ratio_fraction_high:2.045680048468057e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0228400242340285e-05 - rollout_corr/rollout_is_max:2.7041423320770264 - rollout_corr/rollout_is_min:0.4614633619785309 - rollout_corr/rollout_is_std:0.028393879532814026 - rollout_corr/rollout_is_eff_sample_size:0.9991943180123525 - rollout_corr/rollout_is_seq_mean:0.9998977780342102 - rollout_corr/rollout_is_seq_std:0.0016858673188835382 - rollout_corr/rollout_is_seq_max:1.0047959089279175 - rollout_corr/rollout_is_seq_min:0.992766797542572 - rollout_corr/rollout_is_seq_max_deviation:0.0072332024574279785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.305208371952176) - rollout_corr/training_log_ppl:np.float64(0.24501134573074523) - rollout_corr/kl:np.float64(0.0007130547365559181) - rollout_corr/k3_kl:np.float64(0.0005163975527082698) - rollout_corr/rollout_ppl:np.float64(1.3042262229137123) - rollout_corr/rollout_log_ppl:np.float64(0.24429829073051224) - rollout_corr/log_ppl_diff:np.float64(0.0007130550002329983) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015240707798511721) - rollout_corr/log_ppl_diff_max:np.float64(0.009647458791732788) - rollout_corr/log_ppl_diff_min:np.float64(-0.011320427060127258) - rollout_corr/ppl_ratio:np.float64(1.000715468544513) - rollout_corr/chi2_token:np.float64(0.000679319491609931) - rollout_corr/chi2_seq:np.float64(0.30747226788662374) - actor/pg_loss:np.float64(0.0026361719937995076) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025667395266282256) - actor/ppo_kl:np.float64(0.00020520936391221767) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_w_mean:np.float64(5014.568350150017) - self_distillation/token_reweight_w_std:np.float64(103249.80037083285) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.998729647602886) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07348214878584258) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.2412368506193161) - perf/mfu/actor:np.float64(0.050846935709280464) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.622971534729) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.8671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010872789658606052 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010872789658606052 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:381.90234375 - response_length/max:1035.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.90234375 - response_length_non_aborted/max:1035.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002514813095331192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941523926332593) - timing_s/agent_loop/generate_sequences/max:np.float64(8.54596209153533) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.18014469731861) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.54596209153533) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1035 - timing_s/gen:15.545465314760804 - timing_s/reward:0.059370189905166626 - timing_s/old_log_prob:3.289316363632679 - timing_s/adv:0.6208392586559057 - timing_s/update_actor:21.38644278794527 - timing_s/step:40.9937635269016 - timing_s/stop_profile:0.0001326780766248703 - timing_per_token_ms/gen:0.15900524016038953 - timing_per_token_ms/update_actor:0.12800810905510393 - timing_per_token_ms/adv:0.0037160204862358263 - perf/total_num_tokens:167071 - perf/time_per_step:40.9937635269016 - perf/throughput:509.44029538286327 (TaskRunner pid=2320396) Training Progress: 89%|████████▉ | 89/100 [1:24:14<08:09, 44.46s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rearranging to solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the given values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is B: 56 pC. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2320396) step:90 - global_seqlen/min:17689 - global_seqlen/max:24738 - global_seqlen/minmax_diff:7049 - global_seqlen/balanced_min:20971 - global_seqlen/balanced_max:20992 - global_seqlen/mean:20984.875 - actor/entropy:0.21077261865139008 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3020694851875305 - training/rollout_probs_diff_mean:0.002769892103970051 - training/rollout_probs_diff_std:0.008872252888977528 - training/rollout_actor_probs_pearson_corr:0.9991670846939087 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.77734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999796748161316 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8738949298858643 - rollout_corr/rollout_is_min:0.609788715839386 - rollout_corr/rollout_is_std:0.026957370340824127 - rollout_corr/rollout_is_eff_sample_size:0.9992737088438486 - rollout_corr/rollout_is_seq_mean:0.9999539852142334 - rollout_corr/rollout_is_seq_std:0.0014758005272597075 - rollout_corr/rollout_is_seq_max:1.004751205444336 - rollout_corr/rollout_is_seq_min:0.9945812821388245 - rollout_corr/rollout_is_seq_max_deviation:0.005418717861175537 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2639609524048865) - rollout_corr/training_log_ppl:np.float64(0.20912245583895128) - rollout_corr/kl:np.float64(0.0004431889581955062) - rollout_corr/k3_kl:np.float64(0.0004347639480215548) - rollout_corr/rollout_ppl:np.float64(1.2633680049329996) - rollout_corr/rollout_log_ppl:np.float64(0.20867926441133022) - rollout_corr/log_ppl_diff:np.float64(0.0004431914276210591) - rollout_corr/log_ppl_abs_diff:np.float64(0.001261206591152586) - rollout_corr/log_ppl_diff_max:np.float64(0.006049171090126038) - rollout_corr/log_ppl_diff_min:np.float64(-0.005435943603515625) - rollout_corr/ppl_ratio:np.float64(1.0004446292296052) - rollout_corr/chi2_token:np.float64(0.0008622340392321348) - rollout_corr/chi2_seq:np.float64(0.27305450290441513) - actor/pg_loss:np.float64(0.0014611287042498589) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015134929981286405) - actor/ppo_kl:np.float64(2.253385616413084e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_w_mean:np.float64(232.24152618390508) - self_distillation/token_reweight_w_std:np.float64(5283.819025855686) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993226558435708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045578876917716116) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.14182636328041553) - perf/mfu/actor:np.float64(0.05110420901267789) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.68716478347778) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.68046875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17636502357475656) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7504125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13927109847218053) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6064499999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1588835992693667) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.67705) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1764893123600096) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8047875000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09485965224518725) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.54345) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12541633861008067) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7016249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14140393577456317) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8390375000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.061201619852513364) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.493675) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09032488761637099) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7152875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10853346823413709) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8626250000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03995442637984035) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.45824999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05945111823670979) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7266375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08030918738156603) - val-aux/sciknoweval/score/mean@16:np.float64(0.68046875) - val-aux/sciknoweval/score/std@16:np.float64(0.17636502357475656) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7504125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13927109847218053) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6064499999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1588835992693667) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.67705) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1764893123600096) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8047875000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09485965224518725) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.54345) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12541633861008067) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7016249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14140393577456317) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8390375000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.061201619852513364) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.493675) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09032488761637099) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7152875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10853346823413709) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8626250000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03995442637984035) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.45824999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05945111823670979) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7266375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08030918738156603) - val-core/sciknoweval/acc/mean@16:np.float64(0.68046875) - val-aux/sciknoweval/acc/std@16:np.float64(0.17636502357475656) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7504125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13927109847218053) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6064499999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1588835992693667) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.67705) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1764893123600096) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8047875000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09485965224518725) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.54345) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12541633861008067) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7016249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14140393577456317) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8390375000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.061201619852513364) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.493675) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09032488761637099) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7152875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10853346823413709) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8626250000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03995442637984035) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.45824999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05945111823670979) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7266375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08030918738156603) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.035649330156956295) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9954750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013924756097101748) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9727875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.044533748832844) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9836625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036533904346640875) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9990249999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0033521448357730606) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.957075) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05169379024898031) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9919875000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.024094702579460316) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0011135528725660043) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9350624999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.054899137382862784) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.01159806300293624) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9084125000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04978662691013876) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9994375) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00405710688339734) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013788010692223907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013788010692223907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.83984375 - response_length/max:1336.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:388.83984375 - response_length_non_aborted/max:1336.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011170096695423126 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5990730039775372) - timing_s/agent_loop/generate_sequences/max:np.float64(10.28107376769185) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.295882520542364) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.28107376769185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:318 - timing_s/agent_loop/slowest/response_length:1336 - timing_s/gen:16.937936026602983 - timing_s/reward:0.06041034683585167 - timing_s/old_log_prob:2.971561659127474 - timing_s/adv:0.5751631949096918 - timing_s/update_actor:21.22050972469151 - timing_s/step:41.851907912641764 - timing_s/testing:106.41391924209893 - timing_s/save_checkpoint:12.91794090718031 - timing_s/stop_profile:0.00016064755618572235 - timing_per_token_ms/gen:0.1701569776539082 - timing_per_token_ms/update_actor:0.12640359857213535 - timing_per_token_ms/adv:0.0034260580233959684 - perf/total_num_tokens:167879 - perf/time_per_step:41.851907912641764 - perf/throughput:501.40784606049755 (TaskRunner pid=2320396) Training Progress: 90%|█████████ | 90/100 [1:26:55<13:15, 79.50s/it] (TaskRunner pid=2320396) step:91 - global_seqlen/min:18599 - global_seqlen/max:26740 - global_seqlen/minmax_diff:8141 - global_seqlen/balanced_min:21857 - global_seqlen/balanced_max:21878 - global_seqlen/mean:21869.625 - actor/entropy:0.18502238392829895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016291558742523 - training/rollout_probs_diff_mean:0.002582877641543746 - training/rollout_probs_diff_std:0.008864011615514755 - training/rollout_actor_probs_pearson_corr:0.9991196990013123 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.0000965595245361 - rollout_corr/rollout_is_ratio_fraction_high:9.062814569915645e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2347850799560547 - rollout_corr/rollout_is_min:0.5558784604072571 - rollout_corr/rollout_is_std:0.02634231001138687 - rollout_corr/rollout_is_eff_sample_size:0.9993068020008989 - rollout_corr/rollout_is_seq_mean:1.0000598430633545 - rollout_corr/rollout_is_seq_std:0.0019468516111373901 - rollout_corr/rollout_is_seq_max:1.01114022731781 - rollout_corr/rollout_is_seq_min:0.9909047484397888 - rollout_corr/rollout_is_seq_max_deviation:0.011140227317810059 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1936893849633634) - rollout_corr/training_log_ppl:np.float64(0.1685559979378013) - rollout_corr/kl:np.float64(0.00032212701280531775) - rollout_corr/k3_kl:np.float64(0.0004202743424741584) - rollout_corr/rollout_ppl:np.float64(1.1933098202571273) - rollout_corr/rollout_log_ppl:np.float64(0.1682338698883541) - rollout_corr/log_ppl_diff:np.float64(0.0003221280494472012) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013454183208523318) - rollout_corr/log_ppl_diff_max:np.float64(0.010432571172714233) - rollout_corr/log_ppl_diff_min:np.float64(-0.004743427038192749) - rollout_corr/ppl_ratio:np.float64(1.0003239274956286) - rollout_corr/chi2_token:np.float64(0.0010666304733604193) - rollout_corr/chi2_seq:np.float64(0.7431910112500191) - actor/pg_loss:np.float64(0.0014502984704449773) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001065377177837945) - actor/ppo_kl:np.float64(9.484854068375626e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_w_mean:np.float64(88.4975810085889) - self_distillation/token_reweight_w_std:np.float64(2492.829599480203) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999885968863964) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037433789286296815) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.14290286786854267) - perf/mfu/actor:np.float64(0.04989908695015073) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.37800979614258) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003633055603131652 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003633055603131652 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:431.01953125 - response_length/max:1378.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:431.01953125 - response_length_non_aborted/max:1378.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011483579874038696 - timing_s/agent_loop/generate_sequences/min:np.float64(1.07878920994699) - timing_s/agent_loop/generate_sequences/max:np.float64(10.949054036289454) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.725024271836446) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.949054036289454) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:1378 - timing_s/gen:17.278234027326107 - timing_s/reward:0.057705922052264214 - timing_s/old_log_prob:3.1367397028952837 - timing_s/adv:0.6056630592793226 - timing_s/update_actor:22.86518513597548 - timing_s/step:44.04848053306341 - timing_s/stop_profile:0.00012228451669216156 - timing_per_token_ms/gen:0.15658942756841163 - timing_per_token_ms/update_actor:0.13069031325397373 - timing_per_token_ms/adv:0.0034617823766944027 - perf/total_num_tokens:174957 - perf/time_per_step:44.04848053306341 - perf/throughput:496.48988422164416 (TaskRunner pid=2320396) Training Progress: 91%|█████████ | 91/100 [1:27:39<10:19, 68.88s/it] (TaskRunner pid=2320396) step:92 - global_seqlen/min:18621 - global_seqlen/max:25406 - global_seqlen/minmax_diff:6785 - global_seqlen/balanced_min:22801 - global_seqlen/balanced_max:22815 - global_seqlen/mean:22805.625 - actor/entropy:0.2772010862827301 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9653425812721252 - training/rollout_probs_diff_mean:0.003166171023622155 - training/rollout_probs_diff_std:0.009848006069660187 - training/rollout_actor_probs_pearson_corr:0.9991579055786133 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0000650882720947 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6977204470313154e-05 - rollout_corr/rollout_is_max:1.9052884578704834 - rollout_corr/rollout_is_min:0.0294669046998024 - rollout_corr/rollout_is_std:0.029451090842485428 - rollout_corr/rollout_is_eff_sample_size:0.9991336228879825 - rollout_corr/rollout_is_seq_mean:1.0001696348190308 - rollout_corr/rollout_is_seq_std:0.002067823428660631 - rollout_corr/rollout_is_seq_max:1.0114904642105103 - rollout_corr/rollout_is_seq_min:0.9938461184501648 - rollout_corr/rollout_is_seq_max_deviation:0.011490464210510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3618069398216903) - rollout_corr/training_log_ppl:np.float64(0.28330526626086794) - rollout_corr/kl:np.float64(0.0004403062517628342) - rollout_corr/k3_kl:np.float64(0.000519837122439526) - rollout_corr/rollout_ppl:np.float64(1.36120031401515) - rollout_corr/rollout_log_ppl:np.float64(0.28286495895008557) - rollout_corr/log_ppl_diff:np.float64(0.00044030731078237295) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013980344228912145) - rollout_corr/log_ppl_diff_max:np.float64(0.007395714521408081) - rollout_corr/log_ppl_diff_min:np.float64(-0.006245464086532593) - rollout_corr/ppl_ratio:np.float64(1.000442171934992) - rollout_corr/chi2_token:np.float64(0.0012069738004356623) - rollout_corr/chi2_seq:np.float64(0.3731236432213336) - actor/pg_loss:np.float64(0.002277109189890325) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020634635779970267) - actor/ppo_kl:np.float64(9.285395801583718e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(1179.5885476176627) - self_distillation/token_reweight_w_std:np.float64(24205.86808684515) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999315890017897) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06046442501246929) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1943492740392685) - perf/mfu/actor:np.float64(0.053056719866905494) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.21900939941406) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032271479722112417 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032271479722112417 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.39453125 - response_length/max:1221.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.39453125 - response_length_non_aborted/max:1221.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001361556351184845 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0998735800385475) - timing_s/agent_loop/generate_sequences/max:np.float64(10.115503538399935) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7110357238780125) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.115503538399935) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1221 - timing_s/gen:16.898210391402245 - timing_s/reward:0.06314759328961372 - timing_s/old_log_prob:2.9814101308584213 - timing_s/adv:0.6085547208786011 - timing_s/update_actor:22.09982367232442 - timing_s/step:42.74151246063411 - timing_s/stop_profile:0.00012245215475559235 - timing_per_token_ms/gen:0.15195549113261314 - timing_per_token_ms/update_actor:0.1211314295942581 - timing_per_token_ms/adv:0.003335551650517148 - perf/total_num_tokens:182445 - perf/time_per_step:42.74151246063411 - perf/throughput:533.5708468670707 (TaskRunner pid=2320396) Training Progress: 92%|█████████▏| 92/100 [1:28:22<08:08, 61.05s/it] (TaskRunner pid=2320396) step:93 - global_seqlen/min:14223 - global_seqlen/max:23452 - global_seqlen/minmax_diff:9229 - global_seqlen/balanced_min:19017 - global_seqlen/balanced_max:19031 - global_seqlen/mean:19026.25 - actor/entropy:0.27642500400543213 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34021443128585815 - training/rollout_probs_diff_mean:0.003276179078966379 - training/rollout_probs_diff_std:0.009843645617365837 - training/rollout_actor_probs_pearson_corr:0.9992181062698364 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:1.00007164478302 - rollout_corr/rollout_is_ratio_fraction_high:1.1579166312003508e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.3158332624007016e-05 - rollout_corr/rollout_is_max:2.0102672576904297 - rollout_corr/rollout_is_min:0.3711838126182556 - rollout_corr/rollout_is_std:0.02958839386701584 - rollout_corr/rollout_is_eff_sample_size:0.9991255307618554 - rollout_corr/rollout_is_seq_mean:0.9999958872795105 - rollout_corr/rollout_is_seq_std:0.002170138293877244 - rollout_corr/rollout_is_seq_max:1.0074241161346436 - rollout_corr/rollout_is_seq_min:0.9875465035438538 - rollout_corr/rollout_is_seq_max_deviation:0.01245349645614624 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3184588616713881) - rollout_corr/training_log_ppl:np.float64(0.25437020364188356) - rollout_corr/kl:np.float64(0.0006354141595537044) - rollout_corr/k3_kl:np.float64(0.0005414221820103648) - rollout_corr/rollout_ppl:np.float64(1.3176015447825193) - rollout_corr/rollout_log_ppl:np.float64(0.2537347881880123) - rollout_corr/log_ppl_diff:np.float64(0.0006354154538712464) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016181653118110262) - rollout_corr/log_ppl_diff_max:np.float64(0.014689624309539795) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058724284172058105) - rollout_corr/ppl_ratio:np.float64(1.0006382004357874) - rollout_corr/chi2_token:np.float64(0.0008942568674683571) - rollout_corr/chi2_seq:np.float64(0.2074549775570631) - actor/pg_loss:np.float64(0.0034695685608312488) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003058491590763879) - actor/ppo_kl:np.float64(0.0001464155437389536) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(5357.345607595518) - self_distillation/token_reweight_w_std:np.float64(96275.07195161912) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001250740606338) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0668551324197324) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.22902002558112144) - perf/mfu/actor:np.float64(0.04600961155742432) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.25037384033203) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00025763644953258336 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00025763644953258336 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:337.3515625 - response_length/max:1248.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.3515625 - response_length_non_aborted/max:1248.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010005012154579163 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9751391801983118) - timing_s/agent_loop/generate_sequences/max:np.float64(9.368280753493309) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.263870596201741) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.368280753493309) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1248 - timing_s/gen:16.309176307171583 - timing_s/reward:0.059800758957862854 - timing_s/old_log_prob:3.056093478575349 - timing_s/adv:0.5573611091822386 - timing_s/update_actor:21.186741953715682 - timing_s/step:41.25715699791908 - timing_s/stop_profile:2.9325485229492188e-05 - timing_per_token_ms/gen:0.1888466722305132 - timing_per_token_ms/update_actor:0.13919415251110756 - timing_per_token_ms/adv:0.0036617903500574113 - perf/total_num_tokens:152210 - perf/time_per_step:41.25715699791908 - perf/throughput:461.16241118988495 (TaskRunner pid=2320396) Training Progress: 93%|█████████▎| 93/100 [1:29:04<06:25, 55.12s/it] (TaskRunner pid=2320396) step:94 - global_seqlen/min:15297 - global_seqlen/max:24845 - global_seqlen/minmax_diff:9548 - global_seqlen/balanced_min:20488 - global_seqlen/balanced_max:20493 - global_seqlen/mean:20491.75 - actor/entropy:0.20188386738300323 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17276859283447266 - training/rollout_probs_diff_mean:0.002807691926136613 - training/rollout_probs_diff_std:0.009180913679301739 - training/rollout_actor_probs_pearson_corr:0.9991471767425537 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000237226486206 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.5351300239562988 - rollout_corr/rollout_is_std:0.027619199827313423 - rollout_corr/rollout_is_eff_sample_size:0.9992377612073462 - rollout_corr/rollout_is_seq_mean:1.0000065565109253 - rollout_corr/rollout_is_seq_std:0.0017605639295652509 - rollout_corr/rollout_is_seq_max:1.00704026222229 - rollout_corr/rollout_is_seq_min:0.9916078448295593 - rollout_corr/rollout_is_seq_max_deviation:0.008392155170440674 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2156826448626816) - rollout_corr/training_log_ppl:np.float64(0.18709617016429547) - rollout_corr/kl:np.float64(0.0004157023055952891) - rollout_corr/k3_kl:np.float64(0.00043933037052568125) - rollout_corr/rollout_ppl:np.float64(1.2151662367396057) - rollout_corr/rollout_log_ppl:np.float64(0.18668046647508163) - rollout_corr/log_ppl_diff:np.float64(0.00041570368921384215) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013877422898076475) - rollout_corr/log_ppl_diff_max:np.float64(0.00699692964553833) - rollout_corr/log_ppl_diff_min:np.float64(-0.006185248494148254) - rollout_corr/ppl_ratio:np.float64(1.0004174259956926) - rollout_corr/chi2_token:np.float64(0.0009500652085989714) - rollout_corr/chi2_seq:np.float64(0.4689605936873704) - actor/pg_loss:np.float64(0.001823109108954668) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010713179335652967) - actor/ppo_kl:np.float64(9.207377788200688e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(708.5707060070708) - self_distillation/token_reweight_w_std:np.float64(19318.053093866678) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000523213064298) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04706526521476917) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.1663790736347437) - perf/mfu/actor:np.float64(0.049119620321140894) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.20769119262695) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024228731635957956 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024228731635957956 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:376.0546875 - response_length/max:1038.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:376.0546875 - response_length_non_aborted/max:1038.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.8605183362960815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9258252773433924) - timing_s/agent_loop/generate_sequences/max:np.float64(8.57844446413219) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.102110257896129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.57844446413219) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1038 - timing_s/gen:15.732666831463575 - timing_s/reward:0.06416278146207333 - timing_s/old_log_prob:3.082074398174882 - timing_s/adv:0.5425643008202314 - timing_s/update_actor:21.5194071829319 - timing_s/step:41.027859184890985 - timing_s/stop_profile:0.00012390315532684326 - timing_per_token_ms/gen:0.1634223208835938 - timing_per_token_ms/update_actor:0.13126872511456988 - timing_per_token_ms/adv:0.003309650840095596 - perf/total_num_tokens:163934 - perf/time_per_step:41.027859184890985 - perf/throughput:499.4594016630129 (TaskRunner pid=2320396) Training Progress: 94%|█████████▍| 94/100 [1:29:45<05:05, 50.91s/it] (TaskRunner pid=2320396) step:95 - global_seqlen/min:18552 - global_seqlen/max:26721 - global_seqlen/minmax_diff:8169 - global_seqlen/balanced_min:21840 - global_seqlen/balanced_max:21848 - global_seqlen/mean:21846.0 - actor/entropy:0.24933725595474243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2370302975177765 - training/rollout_probs_diff_mean:0.0031582678202539682 - training/rollout_probs_diff_std:0.009481647983193398 - training/rollout_actor_probs_pearson_corr:0.9992415308952332 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000104904174805 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.613905604055617e-06 - rollout_corr/rollout_is_max:1.6393728256225586 - rollout_corr/rollout_is_min:0.27337026596069336 - rollout_corr/rollout_is_std:0.029562193900346756 - rollout_corr/rollout_is_eff_sample_size:0.9991268397733714 - rollout_corr/rollout_is_seq_mean:0.9999439716339111 - rollout_corr/rollout_is_seq_std:0.0017291044350713491 - rollout_corr/rollout_is_seq_max:1.0078022480010986 - rollout_corr/rollout_is_seq_min:0.9936612844467163 - rollout_corr/rollout_is_seq_max_deviation:0.007802248001098633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2998783034272492) - rollout_corr/training_log_ppl:np.float64(0.24693955569819082) - rollout_corr/kl:np.float64(0.0004962469523377422) - rollout_corr/k3_kl:np.float64(0.0005026913720271864) - rollout_corr/rollout_ppl:np.float64(1.2991761877201498) - rollout_corr/rollout_log_ppl:np.float64(0.24644330685259774) - rollout_corr/log_ppl_diff:np.float64(0.0004962488455930725) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013393320696195588) - rollout_corr/log_ppl_diff_max:np.float64(0.0051915645599365234) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062499940395355225) - rollout_corr/ppl_ratio:np.float64(1.0004977483768016) - rollout_corr/chi2_token:np.float64(0.0009992511477321386) - rollout_corr/chi2_seq:np.float64(0.4181784482207149) - actor/pg_loss:np.float64(0.002324818167835474) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000250535831128218) - actor/ppo_kl:np.float64(-3.197742662219838e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(973.5383979976177) - self_distillation/token_reweight_w_std:np.float64(22118.911819823494) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994016685523093) - self_distillation/token_reweight_w_clip_frac:np.float64(0.052211643022019416) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.19546930491924286) - perf/mfu/actor:np.float64(0.05242335858753856) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.08116912841797) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008467447012662888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008467447012662888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.3125 - response_length/max:1203.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.3125 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015277229249477386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1555323153734207) - timing_s/agent_loop/generate_sequences/max:np.float64(9.825576543807983) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.590243221617129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.825576543807983) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1203 - timing_s/gen:16.539687406271696 - timing_s/reward:0.06275773420929909 - timing_s/old_log_prob:2.922587675973773 - timing_s/adv:0.5635931380093098 - timing_s/update_actor:21.523274278268218 - timing_s/step:41.69783091172576 - timing_s/stop_profile:2.7721747756004333e-05 - timing_per_token_ms/gen:0.15901099259990478 - timing_per_token_ms/update_actor:0.12315340496125274 - timing_per_token_ms/adv:0.0032248073904222155 - perf/total_num_tokens:174768 - perf/time_per_step:41.69783091172576 - perf/throughput:523.9121441651952 (TaskRunner pid=2320396) Training Progress: 95%|█████████▌| 95/100 [1:30:26<04:00, 48.15s/it] (TaskRunner pid=2320396) step:96 - global_seqlen/min:15331 - global_seqlen/max:35009 - global_seqlen/minmax_diff:19678 - global_seqlen/balanced_min:23384 - global_seqlen/balanced_max:23414 - global_seqlen/mean:23404.625 - actor/entropy:0.20848210155963898 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28976500034332275 - training/rollout_probs_diff_mean:0.0027812193147838116 - training/rollout_probs_diff_std:0.009169485419988632 - training/rollout_actor_probs_pearson_corr:0.9991740584373474 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0001304149627686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.644163244753145e-06 - rollout_corr/rollout_is_max:1.65871000289917 - rollout_corr/rollout_is_min:0.3381444215774536 - rollout_corr/rollout_is_std:0.027441667392849922 - rollout_corr/rollout_is_eff_sample_size:0.9992477596280513 - rollout_corr/rollout_is_seq_mean:1.0001225471496582 - rollout_corr/rollout_is_seq_std:0.0015613530995324254 - rollout_corr/rollout_is_seq_max:1.006011962890625 - rollout_corr/rollout_is_seq_min:0.9935289025306702 - rollout_corr/rollout_is_seq_max_deviation:0.006471097469329834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2423579399473965) - rollout_corr/training_log_ppl:np.float64(0.20239087475056294) - rollout_corr/kl:np.float64(0.00034805676968829857) - rollout_corr/k3_kl:np.float64(0.0004387744726113141) - rollout_corr/rollout_ppl:np.float64(1.2419147840701044) - rollout_corr/rollout_log_ppl:np.float64(0.20204281566111604) - rollout_corr/log_ppl_diff:np.float64(0.000348059089446906) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012117344085709192) - rollout_corr/log_ppl_diff_max:np.float64(0.007875531911849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.0057569146156311035) - rollout_corr/ppl_ratio:np.float64(1.0003494743723422) - rollout_corr/chi2_token:np.float64(0.0010582820978015661) - rollout_corr/chi2_seq:np.float64(0.7725066640414298) - actor/pg_loss:np.float64(0.0019937818869948387) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022862237619847292) - actor/ppo_kl:np.float64(7.256892491014355e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(18855.210086595966) - self_distillation/token_reweight_w_std:np.float64(398993.84403990314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002333554439247) - self_distillation/token_reweight_w_clip_frac:np.float64(0.047663526580436155) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.16047444380819798) - perf/mfu/actor:np.float64(0.05551098602326122) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.10414505004883) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005147599149495363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005147599149495363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.89453125 - response_length/max:1508.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.89453125 - response_length_non_aborted/max:1508.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5307759642601013e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3833144959062338) - timing_s/agent_loop/generate_sequences/max:np.float64(12.088890012353659) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.130557603857596) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.088890012353659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:1508 - timing_s/gen:18.805637188255787 - timing_s/reward:0.06366230174899101 - timing_s/old_log_prob:2.9904247242957354 - timing_s/adv:0.5898833721876144 - timing_s/update_actor:21.912752106785774 - timing_s/step:44.431054543703794 - timing_s/stop_profile:0.00012172572314739227 - timing_per_token_ms/gen:0.16255899371790455 - timing_per_token_ms/update_actor:0.11703216835767383 - timing_per_token_ms/adv:0.0031504637020867373 - perf/total_num_tokens:187237 - perf/time_per_step:44.431054543703794 - perf/throughput:526.7627617746157 (TaskRunner pid=2320396) Training Progress: 96%|█████████▌| 96/100 [1:31:11<03:08, 47.05s/it] (TaskRunner pid=2320396) step:97 - global_seqlen/min:18399 - global_seqlen/max:32963 - global_seqlen/minmax_diff:14564 - global_seqlen/balanced_min:23676 - global_seqlen/balanced_max:23712 - global_seqlen/mean:23702.25 - actor/entropy:0.2419547587633133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163779854774475 - training/rollout_probs_diff_mean:0.003042820608243346 - training/rollout_probs_diff_std:0.00934920459985733 - training/rollout_actor_probs_pearson_corr:0.9992281794548035 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.84765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.00001859664917 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4824161300784908e-05 - rollout_corr/rollout_is_max:1.64872407913208 - rollout_corr/rollout_is_min:0.289579302072525 - rollout_corr/rollout_is_std:0.028276080265641212 - rollout_corr/rollout_is_eff_sample_size:0.9992009830243436 - rollout_corr/rollout_is_seq_mean:0.9999649524688721 - rollout_corr/rollout_is_seq_std:0.0021841987036168575 - rollout_corr/rollout_is_seq_max:1.009124755859375 - rollout_corr/rollout_is_seq_min:0.9882916808128357 - rollout_corr/rollout_is_seq_max_deviation:0.011708319187164307 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.255678509362042) - rollout_corr/training_log_ppl:np.float64(0.2168065948790172) - rollout_corr/kl:np.float64(0.0004864873664445213) - rollout_corr/k3_kl:np.float64(0.00045971487596574434) - rollout_corr/rollout_ppl:np.float64(1.2550409259274602) - rollout_corr/rollout_log_ppl:np.float64(0.21632010873872787) - rollout_corr/log_ppl_diff:np.float64(0.0004864861402893439) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013925857347203419) - rollout_corr/log_ppl_diff_max:np.float64(0.00998617708683014) - rollout_corr/log_ppl_diff_min:np.float64(-0.006745591759681702) - rollout_corr/ppl_ratio:np.float64(1.0004884300287813) - rollout_corr/chi2_token:np.float64(0.0008650836534798145) - rollout_corr/chi2_seq:np.float64(0.7680268259719014) - actor/pg_loss:np.float64(0.0021499351132661104) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.822644940664759e-05) - actor/ppo_kl:np.float64(-1.3785715332836901e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_w_mean:np.float64(5079.578742446611) - self_distillation/token_reweight_w_std:np.float64(103047.53704692877) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000506963115185) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04400453166454099) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.15901292487978935) - perf/mfu/actor:np.float64(0.05632942768262873) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.07713317871094) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006603226996958256 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006603226996958256 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.0703125 - response_length/max:1746.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.0703125 - response_length_non_aborted/max:1746.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000186799094080925 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9205618463456631) - timing_s/agent_loop/generate_sequences/max:np.float64(12.9535857681185) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.061531551196822) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.9535857681185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1746 - timing_s/gen:20.029064597561955 - timing_s/reward:0.06482006795704365 - timing_s/old_log_prob:3.03081613779068 - timing_s/adv:0.6125223767012358 - timing_s/update_actor:21.9055054243654 - timing_s/step:45.73275777883828 - timing_s/stop_profile:0.00010943599045276642 - timing_per_token_ms/gen:0.16573491599141046 - timing_per_token_ms/update_actor:0.11552439865606325 - timing_per_token_ms/adv:0.003230296578917802 - perf/total_num_tokens:189618 - perf/time_per_step:45.73275777883828 - perf/throughput:518.2772951201215 (TaskRunner pid=2320396) Training Progress: 97%|█████████▋| 97/100 [1:31:57<02:20, 46.67s/it] (TaskRunner pid=2320396) step:98 - global_seqlen/min:18831 - global_seqlen/max:27260 - global_seqlen/minmax_diff:8429 - global_seqlen/balanced_min:22401 - global_seqlen/balanced_max:22407 - global_seqlen/mean:22404.5 - actor/entropy:0.2321578562259674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5020689964294434 - training/rollout_probs_diff_mean:0.0029261824674904346 - training/rollout_probs_diff_std:0.009089733473956585 - training/rollout_actor_probs_pearson_corr:0.9992458820343018 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999915361404419 - rollout_corr/rollout_is_ratio_fraction_high:2.722421777434647e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.074739864445291e-06 - rollout_corr/rollout_is_max:2.632174015045166 - rollout_corr/rollout_is_min:0.4950760304927826 - rollout_corr/rollout_is_std:0.02858218364417553 - rollout_corr/rollout_is_eff_sample_size:0.999183725586957 - rollout_corr/rollout_is_seq_mean:0.999980092048645 - rollout_corr/rollout_is_seq_std:0.001532103167846799 - rollout_corr/rollout_is_seq_max:1.0066100358963013 - rollout_corr/rollout_is_seq_min:0.9930747151374817 - rollout_corr/rollout_is_seq_max_deviation:0.0069252848625183105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2669148072600365) - rollout_corr/training_log_ppl:np.float64(0.22072085265244823) - rollout_corr/kl:np.float64(0.00041741177986587275) - rollout_corr/k3_kl:np.float64(0.0004389282450176779) - rollout_corr/rollout_ppl:np.float64(1.2663823189213872) - rollout_corr/rollout_log_ppl:np.float64(0.22030344046652317) - rollout_corr/log_ppl_diff:np.float64(0.000417412185925059) - rollout_corr/log_ppl_abs_diff:np.float64(0.001250314715434797) - rollout_corr/log_ppl_diff_max:np.float64(0.008546322584152222) - rollout_corr/log_ppl_diff_min:np.float64(-0.0056915283203125) - rollout_corr/ppl_ratio:np.float64(1.0004189505707473) - rollout_corr/chi2_token:np.float64(0.0009120509494096041) - rollout_corr/chi2_seq:np.float64(0.8078791201114655) - actor/pg_loss:np.float64(0.0020201385486871004) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023760700810271373) - actor/ppo_kl:np.float64(-2.3586566145539223e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(1225.9335982890334) - self_distillation/token_reweight_w_std:np.float64(22503.530439826485) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999883871059865) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04507940106850583) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.4081057645380497) - perf/mfu/actor:np.float64(0.05368987949063367) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.05130767822266) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022709534969180822 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022709534969180822 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:430.453125 - response_length/max:1176.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:430.453125 - response_length_non_aborted/max:1176.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015125982463359833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.054880429059267) - timing_s/agent_loop/generate_sequences/max:np.float64(9.709577007219195) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.776259749698511) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.709577007219195) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1176 - timing_s/gen:16.545982444658875 - timing_s/reward:0.06403978727757931 - timing_s/old_log_prob:2.9795034248381853 - timing_s/adv:0.5723136328160763 - timing_s/update_actor:21.520603019744158 - timing_s/step:41.77067055925727 - timing_s/stop_profile:0.00011765211820602417 - timing_per_token_ms/gen:0.15015048136646408 - timing_per_token_ms/update_actor:0.12006852986980382 - timing_per_token_ms/adv:0.0031930730032810163 - perf/total_num_tokens:179236 - perf/time_per_step:41.77067055925727 - perf/throughput:536.3691724368233 (TaskRunner pid=2320396) Training Progress: 98%|█████████▊| 98/100 [1:32:38<01:30, 45.22s/it] (TaskRunner pid=2320396) step:99 - global_seqlen/min:17790 - global_seqlen/max:27332 - global_seqlen/minmax_diff:9542 - global_seqlen/balanced_min:21975 - global_seqlen/balanced_max:21983 - global_seqlen/mean:21978.625 - actor/entropy:0.21217578649520874 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30725976824760437 - training/rollout_probs_diff_mean:0.0027088799979537725 - training/rollout_probs_diff_std:0.009153125807642937 - training/rollout_actor_probs_pearson_corr:0.9991687536239624 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.8828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:1.0000417232513428 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8437087419442832e-05 - rollout_corr/rollout_is_max:1.9659626483917236 - rollout_corr/rollout_is_min:0.05042092502117157 - rollout_corr/rollout_is_std:0.027372073382139206 - rollout_corr/rollout_is_eff_sample_size:0.9992512115102347 - rollout_corr/rollout_is_seq_mean:1.0000988245010376 - rollout_corr/rollout_is_seq_std:0.0017440631054341793 - rollout_corr/rollout_is_seq_max:1.0088425874710083 - rollout_corr/rollout_is_seq_min:0.9949665069580078 - rollout_corr/rollout_is_seq_max_deviation:0.0088425874710083 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2107150228694081) - rollout_corr/training_log_ppl:np.float64(0.1793389076483436) - rollout_corr/kl:np.float64(0.0004552967537865271) - rollout_corr/k3_kl:np.float64(0.00041060264781833666) - rollout_corr/rollout_ppl:np.float64(1.2101408634334803) - rollout_corr/rollout_log_ppl:np.float64(0.1788836106788949) - rollout_corr/log_ppl_diff:np.float64(0.0004552969694486819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012980137325939722) - rollout_corr/log_ppl_diff_max:np.float64(0.007373355329036713) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074315667152404785) - rollout_corr/ppl_ratio:np.float64(1.0004568246658891) - rollout_corr/chi2_token:np.float64(0.0007171817123889923) - rollout_corr/chi2_seq:np.float64(0.3282004913780838) - actor/pg_loss:np.float64(0.001477871905080974) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.6140283504646504e-05) - actor/ppo_kl:np.float64(0.00016689537152991818) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8828125) - self_distillation/token_reweight_w_mean:np.float64(748.2231652874034) - self_distillation/token_reweight_w_std:np.float64(19586.47575551673) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993815938942134) - self_distillation/token_reweight_w_clip_frac:np.float64(0.033909484758623876) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.1121530244126916) - perf/mfu/actor:np.float64(0.05194370252387481) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.09511184692383) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.8828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007213510572910309 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007213510572910309 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:423.73828125 - response_length/max:1213.0 - response_length/min:52.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.73828125 - response_length_non_aborted/max:1213.0 - response_length_non_aborted/min:52.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012556836009025574 - timing_s/agent_loop/generate_sequences/min:np.float64(0.719259187579155) - timing_s/agent_loop/generate_sequences/max:np.float64(9.758743185549974) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.64678174258006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.758743185549974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1213 - timing_s/gen:16.68287494778633 - timing_s/reward:0.06371786259114742 - timing_s/old_log_prob:2.987398575991392 - timing_s/adv:0.5760593675076962 - timing_s/update_actor:21.90517684444785 - timing_s/step:42.30455153994262 - timing_s/stop_profile:3.0780211091041565e-05 - timing_per_token_ms/gen:0.15379181713899104 - timing_per_token_ms/update_actor:0.12458227507662474 - timing_per_token_ms/adv:0.003276247760652089 - perf/total_num_tokens:175829 - perf/time_per_step:42.30455153994262 - perf/throughput:519.5333409751071 (TaskRunner pid=2320396) Training Progress: 99%|█████████▉| 99/100 [1:33:21<00:44, 44.35s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. Using the formula for the electric field due to a point charge: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2320396) step:100 - global_seqlen/min:15293 - global_seqlen/max:27160 - global_seqlen/minmax_diff:11867 - global_seqlen/balanced_min:22042 - global_seqlen/balanced_max:22077 - global_seqlen/mean:22062.125 - actor/entropy:0.2097870260477066 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4133709967136383 - training/rollout_probs_diff_mean:0.0028145588003098965 - training/rollout_probs_diff_std:0.009487012401223183 - training/rollout_actor_probs_pearson_corr:0.9991060495376587 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.953125 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.953125 - rollout_corr/rollout_is_mean:0.9999947547912598 - rollout_corr/rollout_is_ratio_fraction_high:1.8594100765767507e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.789115023915656e-05 - rollout_corr/rollout_is_max:4.497957706451416 - rollout_corr/rollout_is_min:0.3550756275653839 - rollout_corr/rollout_is_std:0.028181063011288643 - rollout_corr/rollout_is_eff_sample_size:0.9992064579221395 - rollout_corr/rollout_is_seq_mean:0.9999136328697205 - rollout_corr/rollout_is_seq_std:0.0020781357306987047 - rollout_corr/rollout_is_seq_max:1.0099788904190063 - rollout_corr/rollout_is_seq_min:0.9891766309738159 - rollout_corr/rollout_is_seq_max_deviation:0.010823369026184082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2158178966492414) - rollout_corr/training_log_ppl:np.float64(0.185322040655592) - rollout_corr/kl:np.float64(0.0004785535343856395) - rollout_corr/k3_kl:np.float64(0.0005256261923278771) - rollout_corr/rollout_ppl:np.float64(1.2151897735893726) - rollout_corr/rollout_log_ppl:np.float64(0.18484348608762957) - rollout_corr/log_ppl_diff:np.float64(0.00047855456796241924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015097330397111364) - rollout_corr/log_ppl_diff_max:np.float64(0.011803552508354187) - rollout_corr/log_ppl_diff_min:np.float64(-0.010602578520774841) - rollout_corr/ppl_ratio:np.float64(1.0004810797981918) - rollout_corr/chi2_token:np.float64(0.0010942460503429174) - rollout_corr/chi2_seq:np.float64(0.5077934148721397) - actor/pg_loss:np.float64(0.0021155468421056867) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039214472985804605) - actor/ppo_kl:np.float64(-6.8683105657108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(395.0223485589959) - self_distillation/token_reweight_w_std:np.float64(8410.495676852704) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012438350822777) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05156318433000706) - self_distillation/empty_target_batch:np.float64(0.046875) - actor/grad_norm:np.float64(0.16905477456748486) - perf/mfu/actor:np.float64(0.05262459257753069) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.11225509643555) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6578125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1791601964104201) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.72985) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14243945466552344) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5846375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16119285610681686) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.657775) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17855468574283387) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7821) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09939721158640422) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5175124999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12756100353900784) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6794749999999998) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13936384594983817) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8189875000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06830858426455963) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4656625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08893811125770881) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6921250000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09885195998023369) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8481500000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04638693035505402) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4310375000000001) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04764229560029223) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.70175) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06840959900793887) - val-aux/sciknoweval/score/mean@16:np.float64(0.6578125) - val-aux/sciknoweval/score/std@16:np.float64(0.1791601964104201) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.72985) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14243945466552344) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5846375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16119285610681686) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.657775) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17855468574283387) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7821) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09939721158640422) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5175124999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12756100353900784) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6794749999999998) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13936384594983817) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8189875000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06830858426455963) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4656625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08893811125770881) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6921250000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09885195998023369) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8481500000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04638693035505402) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4310375000000001) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04764229560029223) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.70175) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06840959900793887) - val-core/sciknoweval/acc/mean@16:np.float64(0.6578125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1791601964104201) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.72985) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14243945466552344) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5846375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16119285610681686) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.657775) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17855468574283387) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7821) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09939721158640422) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5175124999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12756100353900784) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6794749999999998) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13936384594983817) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8189875000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06830858426455963) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4656625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08893811125770881) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6921250000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09885195998023369) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8481500000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04638693035505402) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4310375000000001) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04764229560029223) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.70175) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06840959900793887) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.036863588899184146) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9937625000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.018150023927969855) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9683875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04360969010040857) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9808749999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03671858855568414) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9987124999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.005042814272451763) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9510124999999998) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.047638230389618993) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.990175) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.024837100384058737) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0008816709987291178) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9298375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04533008795714807) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9957125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.013499621923874276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.90865) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03526981610931457) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9989250000000002) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004970325714929565) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0074690175242722034 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0074690175242722034 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.16015625 - response_length/max:1576.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.16015625 - response_length_non_aborted/max:1576.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5948509573936462e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9642361830919981) - timing_s/agent_loop/generate_sequences/max:np.float64(11.51597017236054) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3494094031411805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.51597017236054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1576 - timing_s/gen:18.236607283353806 - timing_s/reward:0.06037502735853195 - timing_s/old_log_prob:2.968335896730423 - timing_s/adv:0.5751208234578371 - timing_s/update_actor:21.881923211738467 - timing_s/step:43.809186674654484 - timing_s/testing:113.72614772059023 - timing_s/save_checkpoint:12.852769210934639 - timing_s/stop_profile:0.0001350194215774536 - timing_per_token_ms/gen:0.16954665058296042 - timing_per_token_ms/update_actor:0.12397900934145321 - timing_per_token_ms/adv:0.003258530306225245 - perf/total_num_tokens:176497 - perf/time_per_step:43.809186674654484 - perf/throughput:503.5958591023991 (TaskRunner pid=2320396) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2320396) "np.float64(0.6578125), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2320396) "np.float64(0.1791601964104201), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2320396) "np.float64(0.72985), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2320396) "np.float64(0.14243945466552344), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2320396) "np.float64(0.5846375), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2320396) "np.float64(0.16119285610681686), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2320396) "np.float64(0.657775), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2320396) "np.float64(0.17855468574283387), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2320396) "np.float64(0.7821), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2320396) "np.float64(0.09939721158640422), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2320396) "np.float64(0.5175124999999999), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2320396) "np.float64(0.12756100353900784), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2320396) "np.float64(0.6794749999999998), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2320396) "np.float64(0.13936384594983817), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2320396) "np.float64(0.8189875000000001), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2320396) "np.float64(0.06830858426455963), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2320396) "np.float64(0.4656625), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2320396) "np.float64(0.08893811125770881), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2320396) "np.float64(0.6921250000000001), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2320396) "np.float64(0.09885195998023369), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2320396) "np.float64(0.8481500000000001), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2320396) "np.float64(0.04638693035505402), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2320396) "np.float64(0.4310375000000001), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2320396) "np.float64(0.04764229560029223), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2320396) "np.float64(0.70175), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2320396) "np.float64(0.06840959900793887), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2320396) "np.float64(0.6578125), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2320396) "np.float64(0.1791601964104201), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2320396) "np.float64(0.72985), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2320396) "np.float64(0.14243945466552344), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2320396) "np.float64(0.5846375), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2320396) "np.float64(0.16119285610681686), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2320396) "np.float64(0.657775), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2320396) "np.float64(0.17855468574283387), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2320396) "np.float64(0.7821), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2320396) "np.float64(0.09939721158640422), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2320396) "np.float64(0.5175124999999999), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2320396) "np.float64(0.12756100353900784), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2320396) "np.float64(0.6794749999999998), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2320396) "np.float64(0.13936384594983817), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2320396) "np.float64(0.8189875000000001), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2320396) "np.float64(0.06830858426455963), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2320396) "np.float64(0.4656625), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2320396) "np.float64(0.08893811125770881), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2320396) "np.float64(0.6921250000000001), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2320396) "np.float64(0.09885195998023369), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2320396) "np.float64(0.8481500000000001), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2320396) "np.float64(0.04638693035505402), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2320396) "np.float64(0.4310375000000001), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2320396) "np.float64(0.04764229560029223), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2320396) "np.float64(0.70175), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2320396) "np.float64(0.06840959900793887), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2320396) "np.float64(0.6578125), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2320396) "np.float64(0.1791601964104201), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2320396) "np.float64(0.72985), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2320396) "np.float64(0.14243945466552344), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2320396) "np.float64(0.5846375), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2320396) "np.float64(0.16119285610681686), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2320396) "np.float64(0.657775), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2320396) "np.float64(0.17855468574283387), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2320396) "np.float64(0.7821), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2320396) "np.float64(0.09939721158640422), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2320396) "np.float64(0.5175124999999999), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2320396) "np.float64(0.12756100353900784), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2320396) "np.float64(0.6794749999999998), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2320396) "np.float64(0.13936384594983817), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2320396) "np.float64(0.8189875000000001), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2320396) "np.float64(0.06830858426455963), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2320396) "np.float64(0.4656625), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2320396) "np.float64(0.08893811125770881), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2320396) "np.float64(0.6921250000000001), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2320396) "np.float64(0.09885195998023369), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2320396) "np.float64(0.8481500000000001), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2320396) "np.float64(0.04638693035505402), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2320396) "np.float64(0.4310375000000001), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2320396) "np.float64(0.04764229560029223), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2320396) "np.float64(0.70175), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2320396) 'np.float64(0.06840959900793887), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.98125), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2320396) 'np.float64(0.036863588899184146), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2320396) 'np.float64(0.9937625000000001), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2320396) 'np.float64(0.018150023927969855), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2320396) 'np.float64(0.9683875000000001), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2320396) 'np.float64(0.04360969010040857), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': " (TaskRunner pid=2320396) 'np.float64(0.9808749999999999), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2320396) 'np.float64(0.03671858855568414), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2320396) 'np.float64(0.9987124999999999), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2320396) 'np.float64(0.005042814272451763), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2320396) 'np.float64(0.9510124999999998), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2320396) 'np.float64(0.047638230389618993), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.990175), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2320396) 'np.float64(0.024837100384058737), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(0.9999375), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@8/std': " (TaskRunner pid=2320396) 'np.float64(0.0008816709987291178), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.9298375), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2320396) 'np.float64(0.04533008795714807), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(0.9957125), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2320396) 'np.float64(0.013499621923874276), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(0.90865), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2320396) 'np.float64(0.03526981610931457), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2320396) 'np.float64(0.9989250000000002), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2320396) "np.float64(0.004970325714929565), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2320396) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2320396) 'np.float64(2.0)}') (TaskRunner pid=2320396) Training Progress: 100%|██████████| 100/100 [1:36:11<00:00, 82.17s/it] Training Progress: 100%|██████████| 100/100 [1:36:11<00:00, 57.72s/it] (TaskRunner pid=2320396) Exception ignored in: (TaskRunner pid=2320396) Traceback (most recent call last): (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/threading.py", line 1594, in _shutdown (TaskRunner pid=2320396) atexit_call() (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/process.py", line 101, in _python_exit (TaskRunner pid=2320396) items = list(_threads_wakeups.items()) (TaskRunner pid=2320396) Exception ignored in: (TaskRunner pid=2320396) ^^Traceback (most recent call last): (TaskRunner pid=2320396) ^^ File "/mnt/mole/SDPO/L2T/verl/utils/tracking.py", line 169, in __del__ (TaskRunner pid=2320396) ^^^^^ ^self.logger["wandb"].finish(exit_code=0)^ (TaskRunner pid=2320396) ^^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 4097, in finish (TaskRunner pid=2320396) ^^^^ ^wandb.run.finish(exit_code=exit_code, quiet=quiet)^ (TaskRunner pid=2320396) ^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2320396) ^^ ^return func(self, *args, **kwargs)^ (TaskRunner pid=2320396) ^ ^ ^ ^ ^ ^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/weakref.py", line 462, in items (TaskRunner pid=2320396) ^^^^^^^^^^^ ^with _IterationGuard(self):^ (TaskRunner pid=2320396) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^^^^^^^^^^ (TaskRunner pid=2320396) ^^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 445, in wrapper (TaskRunner pid=2320396) ^^ ^return func(self, *args, **kwargs)^ (TaskRunner pid=2320396) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/_weakrefset.py", line 31, in __exit__ (TaskRunner pid=2320396) ^^^^^^^^^^^^^ ^s.remove(self)^ (TaskRunner pid=2320396) ^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2320396) ^^^^^^ ^_error_if_any_worker_fails()^ (TaskRunner pid=2320396) ^RuntimeError^: ^DataLoader worker (pid 2342222) is killed by signal: Killed. (TaskRunner pid=2320396) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2275, in finish (TaskRunner pid=2320396) return self._finish(exit_code) (TaskRunner pid=2320396) ^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2320396) return func(self, *args, **kwargs) (TaskRunner pid=2320396) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2288, in _finish (TaskRunner pid=2320396) with telemetry.context(run=self) as tel: (TaskRunner pid=2320396) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/telemetry.py", line 42, in __exit__ (TaskRunner pid=2320396) self._run._telemetry_callback(self._obj) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 752, in _telemetry_callback (TaskRunner pid=2320396) self._telemetry_flush() (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 765, in _telemetry_flush (TaskRunner pid=2320396) self._backend.interface._publish_telemetry(self._telemetry_obj) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_shared.py", line 103, in _publish_telemetry (TaskRunner pid=2320396) self._publish(rec) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 46, in _publish (TaskRunner pid=2320396) self._asyncer.run(lambda: self._client.publish(request)) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 131, in run (TaskRunner pid=2320396) raise ValueError("Cannot use run() inside async loop.") (TaskRunner pid=2320396) ValueError: Cannot use run() inside async loop. main_ppo exit code: 0 [2026-07-03 07:27:53] Finished Qwen/Qwen3-8B physics rlsd_tr. [2026-07-03 07:27:53] Starting Qwen/Qwen3-8B biology grpo. Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_grpo_Qwen3_8B 2026-07-03 07:27:55,383 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 07:28:05,194 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2347384) TaskRunner hostname: mole-workspace-rw, PID: 2347384 (TaskRunner pid=2347384) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2347384) 'calculate_entropy': False, (TaskRunner pid=2347384) 'calculate_sum_pi_squared': False, (TaskRunner pid=2347384) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2347384) 'async_save': False, (TaskRunner pid=2347384) 'load_contents': ['model', (TaskRunner pid=2347384) 'optimizer', (TaskRunner pid=2347384) 'extra'], (TaskRunner pid=2347384) 'save_contents': ['model', (TaskRunner pid=2347384) 'optimizer', (TaskRunner pid=2347384) 'extra']}, (TaskRunner pid=2347384) 'clip_ratio': 0.2, (TaskRunner pid=2347384) 'clip_ratio_c': 3.0, (TaskRunner pid=2347384) 'clip_ratio_high': 0.28, (TaskRunner pid=2347384) 'clip_ratio_low': 0.2, (TaskRunner pid=2347384) 'data_loader_seed': 42, (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_coeff': 0, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'freeze_vision_tower': False, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'forward_only': False, (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'full_determinism': False, (TaskRunner pid=2347384) 'model_dtype': 'fp32', (TaskRunner pid=2347384) 'offload_policy': False, (TaskRunner pid=2347384) 'optimizer_offload': False, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'seed': 42, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_orig_params': False, (TaskRunner pid=2347384) 'use_torch_compile': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'grad_clip': 1.0, (TaskRunner pid=2347384) 'kl_loss_coef': 0.0, (TaskRunner pid=2347384) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2347384) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2347384) 'loss_scale_factor': None, (TaskRunner pid=2347384) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2347384) 'betas': [0.9, 0.999], (TaskRunner pid=2347384) 'clip_grad': 1.0, (TaskRunner pid=2347384) 'lr': 1e-06, (TaskRunner pid=2347384) 'lr_scheduler_type': 'constant', (TaskRunner pid=2347384) 'lr_warmup_steps': 10, (TaskRunner pid=2347384) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2347384) 'min_lr_ratio': 0.0, (TaskRunner pid=2347384) 'num_cycles': 0.5, (TaskRunner pid=2347384) 'optimizer': 'AdamW', (TaskRunner pid=2347384) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2347384) 'override_optimizer_config': None, (TaskRunner pid=2347384) 'total_training_steps': -1, (TaskRunner pid=2347384) 'warmup_style': None, (TaskRunner pid=2347384) 'weight_decay': 0.01}, (TaskRunner pid=2347384) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2347384) 'clip_cov_lb': 1.0, (TaskRunner pid=2347384) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2347384) 'clip_cov_ub': 5.0, (TaskRunner pid=2347384) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2347384) 'loss_mode': 'vanilla', (TaskRunner pid=2347384) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2347384) 'ppo_epochs': 1, (TaskRunner pid=2347384) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2347384) 'ppo_micro_batch_size': None, (TaskRunner pid=2347384) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2347384) 'ppo_mini_batch_size': 8, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'rollout_n': 8, (TaskRunner pid=2347384) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2347384) 'mode': 'disabled', (TaskRunner pid=2347384) 'record_file': None, (TaskRunner pid=2347384) 'replay_file': None}, (TaskRunner pid=2347384) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2347384) 'alpha': 0.5, (TaskRunner pid=2347384) 'distillation_add_tail': True, (TaskRunner pid=2347384) 'distillation_topk': 100, (TaskRunner pid=2347384) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2347384) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2347384) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'loss_weight': 0.0, (TaskRunner pid=2347384) 'mask': 'all'}, (TaskRunner pid=2347384) 'feedback_template': '\n' (TaskRunner pid=2347384) 'The ' (TaskRunner pid=2347384) 'following ' (TaskRunner pid=2347384) 'is ' (TaskRunner pid=2347384) 'feedback ' (TaskRunner pid=2347384) 'from ' (TaskRunner pid=2347384) 'your ' (TaskRunner pid=2347384) 'unsuccessful ' (TaskRunner pid=2347384) 'earlier ' (TaskRunner pid=2347384) 'attempt:\n' (TaskRunner pid=2347384) '\n' (TaskRunner pid=2347384) '{feedback_raw}', (TaskRunner pid=2347384) 'full_logit_distillation': True, (TaskRunner pid=2347384) 'include_environment_feedback': True, (TaskRunner pid=2347384) 'is_clip': 2, (TaskRunner pid=2347384) 'max_reprompt_len': 22528, (TaskRunner pid=2347384) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2347384) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2347384) '\n' (TaskRunner pid=2347384) 'Correctly ' (TaskRunner pid=2347384) 'solve ' (TaskRunner pid=2347384) 'the ' (TaskRunner pid=2347384) 'original ' (TaskRunner pid=2347384) 'question.', (TaskRunner pid=2347384) 'reprompt_truncation': 'right', (TaskRunner pid=2347384) 'solution_template': '\n' (TaskRunner pid=2347384) 'Correct ' (TaskRunner pid=2347384) 'solution:\n' (TaskRunner pid=2347384) '\n' (TaskRunner pid=2347384) '{successful_previous_attempt}', (TaskRunner pid=2347384) 'srpo_dynamic_weighting': False, (TaskRunner pid=2347384) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2347384) 'success_reward_threshold': 0.5, (TaskRunner pid=2347384) 'teacher_regularization': 'ema', (TaskRunner pid=2347384) 'teacher_update_rate': 0.0, (TaskRunner pid=2347384) 'token_reweight_decay_steps': None, (TaskRunner pid=2347384) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2347384) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2347384) 'shuffle': False, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2347384) 'tau_neg': 1.05, (TaskRunner pid=2347384) 'tau_pos': 1.0, (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_dynamic_bsz': False, (TaskRunner pid=2347384) 'use_fused_kernels': False, (TaskRunner pid=2347384) 'use_kl_loss': False, (TaskRunner pid=2347384) 'use_prefix_grouper': False, (TaskRunner pid=2347384) 'use_remove_padding': True, (TaskRunner pid=2347384) 'use_torch_compile': True}, (TaskRunner pid=2347384) 'hybrid_engine': True, (TaskRunner pid=2347384) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2347384) 'custom_chat_template': None, (TaskRunner pid=2347384) 'enable_activation_offload': False, (TaskRunner pid=2347384) 'enable_gradient_checkpointing': True, (TaskRunner pid=2347384) 'exclude_modules': None, (TaskRunner pid=2347384) 'external_lib': None, (TaskRunner pid=2347384) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2347384) 'hf_config_path': None, (TaskRunner pid=2347384) 'lora_adapter_path': None, (TaskRunner pid=2347384) 'lora_alpha': 16, (TaskRunner pid=2347384) 'lora_rank': 0, (TaskRunner pid=2347384) 'override_config': {}, (TaskRunner pid=2347384) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'target_modules': 'all-linear', (TaskRunner pid=2347384) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2347384) 'num_shards': 4}, (TaskRunner pid=2347384) 'tokenizer_path': None, (TaskRunner pid=2347384) 'trust_remote_code': True, (TaskRunner pid=2347384) 'use_fused_kernels': False, (TaskRunner pid=2347384) 'use_liger': False, (TaskRunner pid=2347384) 'use_remove_padding': True, (TaskRunner pid=2347384) 'use_shm': False}, (TaskRunner pid=2347384) 'nccl_timeout': 600, (TaskRunner pid=2347384) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'forward_only': True, (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'full_determinism': False, (TaskRunner pid=2347384) 'model_dtype': 'fp32', (TaskRunner pid=2347384) 'offload_policy': False, (TaskRunner pid=2347384) 'optimizer_offload': False, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'seed': 42, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_orig_params': False, (TaskRunner pid=2347384) 'use_torch_compile': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2347384) 'log_prob_micro_batch_size': None, (TaskRunner pid=2347384) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2347384) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'rollout_n': 8, (TaskRunner pid=2347384) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2347384) 'mode': 'disabled', (TaskRunner pid=2347384) 'record_file': None, (TaskRunner pid=2347384) 'replay_file': None}, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_torch_compile': True}, (TaskRunner pid=2347384) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2347384) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2347384) 'agent_loop_config_path': None, (TaskRunner pid=2347384) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2347384) 'name': None, (TaskRunner pid=2347384) 'path': None}, (TaskRunner pid=2347384) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2347384) 'num_workers': 8}, (TaskRunner pid=2347384) 'calculate_log_probs': True, (TaskRunner pid=2347384) 'cudagraph_capture_sizes': None, (TaskRunner pid=2347384) 'data_parallel_size': 1, (TaskRunner pid=2347384) 'disable_log_stats': True, (TaskRunner pid=2347384) 'do_sample': True, (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'enable_chunked_prefill': True, (TaskRunner pid=2347384) 'enable_prefix_caching': True, (TaskRunner pid=2347384) 'enable_rollout_routing_replay': False, (TaskRunner pid=2347384) 'enforce_eager': False, (TaskRunner pid=2347384) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2347384) 'expert_parallel_size': 1, (TaskRunner pid=2347384) 'free_cache_engine': True, (TaskRunner pid=2347384) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2347384) 'ignore_eos': False, (TaskRunner pid=2347384) 'layered_summon': False, (TaskRunner pid=2347384) 'load_format': 'dummy', (TaskRunner pid=2347384) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2347384) 'log_prob_micro_batch_size': None, (TaskRunner pid=2347384) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2347384) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2347384) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2347384) 'max_model_len': 10240, (TaskRunner pid=2347384) 'max_num_batched_tokens': 10240, (TaskRunner pid=2347384) 'max_num_seqs': 1024, (TaskRunner pid=2347384) 'mode': 'async', (TaskRunner pid=2347384) 'multi_stage_wake_up': False, (TaskRunner pid=2347384) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'format': 'hermes', (TaskRunner pid=2347384) 'interaction_config_path': None, (TaskRunner pid=2347384) 'max_assistant_turns': None, (TaskRunner pid=2347384) 'max_parallel_calls': 1, (TaskRunner pid=2347384) 'max_tool_response_length': 256, (TaskRunner pid=2347384) 'max_user_turns': None, (TaskRunner pid=2347384) 'num_repeat_rollouts': None, (TaskRunner pid=2347384) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2347384) 'tool_config_path': None, (TaskRunner pid=2347384) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2347384) 'use_inference_chat_template': False}, (TaskRunner pid=2347384) 'n': 8, (TaskRunner pid=2347384) 'name': 'vllm', (TaskRunner pid=2347384) 'over_sample_rate': 0, (TaskRunner pid=2347384) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2347384) 'port': 9090, (TaskRunner pid=2347384) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2347384) 'prompt_length': 2048, (TaskRunner pid=2347384) 'quantization': None, (TaskRunner pid=2347384) 'quantization_config_file': None, (TaskRunner pid=2347384) 'response_length': 8192, (TaskRunner pid=2347384) 'scheduling_policy': 'fcfs', (TaskRunner pid=2347384) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2347384) 'skip_rollout': False, (TaskRunner pid=2347384) 'skip_tokenizer_init': True, (TaskRunner pid=2347384) 'temperature': 1.0, (TaskRunner pid=2347384) 'tensor_model_parallel_size': 2, (TaskRunner pid=2347384) 'top_k': -1, (TaskRunner pid=2347384) 'top_p': 1.0, (TaskRunner pid=2347384) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2347384) 'backend': None, (TaskRunner pid=2347384) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2347384) 'token2text': False}, (TaskRunner pid=2347384) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2347384) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2347384) 'do_sample': True, (TaskRunner pid=2347384) 'n': 16, (TaskRunner pid=2347384) 'temperature': 0.6, (TaskRunner pid=2347384) 'top_k': -1, (TaskRunner pid=2347384) 'top_p': 0.95}}}, (TaskRunner pid=2347384) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2347384) 'adv_estimator': 'grpo', (TaskRunner pid=2347384) 'gamma': 1.0, (TaskRunner pid=2347384) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2347384) 'horizon': 10000, (TaskRunner pid=2347384) 'kl_coef': 0.001, (TaskRunner pid=2347384) 'target_kl': 0.1, (TaskRunner pid=2347384) 'type': 'fixed'}, (TaskRunner pid=2347384) 'kl_penalty': 'kl', (TaskRunner pid=2347384) 'lam': 1.0, (TaskRunner pid=2347384) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2347384) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2347384) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2347384) 'loss_type': 'ppo_clip', (TaskRunner pid=2347384) 'rollout_is': 'token', (TaskRunner pid=2347384) 'rollout_is_batch_normalize': False, (TaskRunner pid=2347384) 'rollout_is_threshold': 2.0, (TaskRunner pid=2347384) 'rollout_rs': None, (TaskRunner pid=2347384) 'rollout_rs_threshold': None}, (TaskRunner pid=2347384) 'use_kl_in_reward': False, (TaskRunner pid=2347384) 'use_pf_ppo': False}, (TaskRunner pid=2347384) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2347384) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2347384) 'async_save': False, (TaskRunner pid=2347384) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2347384) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2347384) 'cliprange_value': 0.5, (TaskRunner pid=2347384) 'data_loader_seed': 42, (TaskRunner pid=2347384) 'enable': None, (TaskRunner pid=2347384) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2347384) 'forward_micro_batch_size': None, (TaskRunner pid=2347384) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2347384) 'grad_clip': 1.0, (TaskRunner pid=2347384) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2347384) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2347384) 'enable_activation_offload': False, (TaskRunner pid=2347384) 'enable_gradient_checkpointing': True, (TaskRunner pid=2347384) 'external_lib': None, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'forward_only': False, (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'full_determinism': False, (TaskRunner pid=2347384) 'model_dtype': 'fp32', (TaskRunner pid=2347384) 'offload_policy': False, (TaskRunner pid=2347384) 'optimizer_offload': False, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'seed': 42, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_orig_params': False, (TaskRunner pid=2347384) 'use_torch_compile': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'lora_alpha': 16, (TaskRunner pid=2347384) 'lora_rank': 0, (TaskRunner pid=2347384) 'override_config': {}, (TaskRunner pid=2347384) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'target_modules': 'all-linear', (TaskRunner pid=2347384) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2347384) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'trust_remote_code': True, (TaskRunner pid=2347384) 'use_remove_padding': False, (TaskRunner pid=2347384) 'use_shm': False}, (TaskRunner pid=2347384) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2347384) 'betas': [0.9, 0.999], (TaskRunner pid=2347384) 'clip_grad': 1.0, (TaskRunner pid=2347384) 'lr': 1e-05, (TaskRunner pid=2347384) 'lr_scheduler_type': 'constant', (TaskRunner pid=2347384) 'lr_warmup_steps': -1, (TaskRunner pid=2347384) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2347384) 'min_lr_ratio': 0.0, (TaskRunner pid=2347384) 'num_cycles': 0.5, (TaskRunner pid=2347384) 'optimizer': 'AdamW', (TaskRunner pid=2347384) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2347384) 'override_optimizer_config': None, (TaskRunner pid=2347384) 'total_training_steps': -1, (TaskRunner pid=2347384) 'warmup_style': None, (TaskRunner pid=2347384) 'weight_decay': 0.01}, (TaskRunner pid=2347384) 'ppo_epochs': 1, (TaskRunner pid=2347384) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2347384) 'ppo_micro_batch_size': None, (TaskRunner pid=2347384) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2347384) 'ppo_mini_batch_size': 8, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'rollout_n': 8, (TaskRunner pid=2347384) 'shuffle': False, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_dynamic_bsz': False}, (TaskRunner pid=2347384) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2347384) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2347384) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2347384) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2347384) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2347384) 'dataloader_num_workers': 8, (TaskRunner pid=2347384) 'filter_overlong_prompts': True, (TaskRunner pid=2347384) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2347384) 'image_key': 'images', (TaskRunner pid=2347384) 'image_patch_size': 14, (TaskRunner pid=2347384) 'max_prompt_length': 2048, (TaskRunner pid=2347384) 'max_response_length': 8192, (TaskRunner pid=2347384) 'prompt_key': 'prompt', (TaskRunner pid=2347384) 'return_full_prompt': False, (TaskRunner pid=2347384) 'return_multi_modal_inputs': True, (TaskRunner pid=2347384) 'return_raw_chat': True, (TaskRunner pid=2347384) 'return_raw_input_ids': False, (TaskRunner pid=2347384) 'reward_fn_key': 'data_source', (TaskRunner pid=2347384) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2347384) 'seed': None, (TaskRunner pid=2347384) 'shuffle': True, (TaskRunner pid=2347384) 'tokenizer': None, (TaskRunner pid=2347384) 'tool_config_path': None, (TaskRunner pid=2347384) 'train_batch_size': 32, (TaskRunner pid=2347384) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2347384) 'train_max_samples': 3200, (TaskRunner pid=2347384) 'truncation': 'error', (TaskRunner pid=2347384) 'trust_remote_code': True, (TaskRunner pid=2347384) 'use_shm': False, (TaskRunner pid=2347384) 'val_batch_size': None, (TaskRunner pid=2347384) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2347384) 'val_max_samples': -1, (TaskRunner pid=2347384) 'validation_shuffle': False, (TaskRunner pid=2347384) 'video_key': 'videos'}, (TaskRunner pid=2347384) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2347384) 'cuda-memory-usage': 'true', (TaskRunner pid=2347384) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2347384) 'capture-range-end': None, (TaskRunner pid=2347384) 'cuda-graph-trace': 'graph', (TaskRunner pid=2347384) 'cuda-memory-usage': 'true', (TaskRunner pid=2347384) 'kill': 'none', (TaskRunner pid=2347384) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2347384) 'torch_memory': {'context': 'all', (TaskRunner pid=2347384) 'kw_args': {}, (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'stacks': 'all', (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2347384) 'profile_continuous_steps': False, (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'steps': None, (TaskRunner pid=2347384) 'tool': None}, (TaskRunner pid=2347384) 'max_model_len': 10240, (TaskRunner pid=2347384) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_8B', (TaskRunner pid=2347384) 'include_dashboard': False, (TaskRunner pid=2347384) 'num_cpus': None, (TaskRunner pid=2347384) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2347384) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2347384) 'USER': 'root'}}}, (TaskRunner pid=2347384) 'timeline_json_file': None}, (TaskRunner pid=2347384) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2347384) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2347384) 'name': 'custom_reward_manager', (TaskRunner pid=2347384) 'path': None}, (TaskRunner pid=2347384) 'name': 'naive', (TaskRunner pid=2347384) 'source': 'register'}, (TaskRunner pid=2347384) 'reward_model': {'enable': False, (TaskRunner pid=2347384) 'enable_resource_pool': False, (TaskRunner pid=2347384) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2347384) 'launch_reward_fn_async': False, (TaskRunner pid=2347384) 'max_length': None, (TaskRunner pid=2347384) 'micro_batch_size': None, (TaskRunner pid=2347384) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2347384) 'model': {'external_lib': None, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'override_config': {}, (TaskRunner pid=2347384) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2347384) 'trust_remote_code': False, (TaskRunner pid=2347384) 'use_fused_kernels': False, (TaskRunner pid=2347384) 'use_remove_padding': False, (TaskRunner pid=2347384) 'use_shm': False}, (TaskRunner pid=2347384) 'n_gpus_per_node': 8, (TaskRunner pid=2347384) 'nnodes': 0, (TaskRunner pid=2347384) 'num_workers': 1, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'reward_loop_class_name': None, (TaskRunner pid=2347384) 'reward_loop_module_path': None, (TaskRunner pid=2347384) 'reward_loop_source': 'register', (TaskRunner pid=2347384) 'reward_manager': 'naive', (TaskRunner pid=2347384) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2347384) 'cudagraph_capture_sizes': None, (TaskRunner pid=2347384) 'data_parallel_size': 1, (TaskRunner pid=2347384) 'disable_log_stats': True, (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'enable_chunked_prefill': True, (TaskRunner pid=2347384) 'enable_prefix_caching': True, (TaskRunner pid=2347384) 'enforce_eager': True, (TaskRunner pid=2347384) 'engine_kwargs': {}, (TaskRunner pid=2347384) 'expert_parallel_size': 1, (TaskRunner pid=2347384) 'free_cache_engine': True, (TaskRunner pid=2347384) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2347384) 'limit_images': None, (TaskRunner pid=2347384) 'load_format': 'auto', (TaskRunner pid=2347384) 'max_model_len': None, (TaskRunner pid=2347384) 'max_num_batched_tokens': 8192, (TaskRunner pid=2347384) 'max_num_seqs': 1024, (TaskRunner pid=2347384) 'name': '???', (TaskRunner pid=2347384) 'prompt_length': 2048, (TaskRunner pid=2347384) 'response_length': 2048, (TaskRunner pid=2347384) 'skip_tokenizer_init': False, (TaskRunner pid=2347384) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2347384) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2347384) 'memory_limit_mb': 1024, (TaskRunner pid=2347384) 'url': None}, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_dynamic_bsz': False, (TaskRunner pid=2347384) 'use_reward_loop': False}, (TaskRunner pid=2347384) 'trainer': {'balance_batch': True, (TaskRunner pid=2347384) 'critic_warmup': 0, (TaskRunner pid=2347384) 'default_hdfs_dir': None, (TaskRunner pid=2347384) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2347384) 'del_local_ckpt_after_load': False, (TaskRunner pid=2347384) 'device': 'cuda', (TaskRunner pid=2347384) 'esi_redundant_time': 0, (TaskRunner pid=2347384) 'experiment_name': 'qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2347384) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2347384) 'log_val_generations': 0, (TaskRunner pid=2347384) 'logger': ['console', 'wandb'], (TaskRunner pid=2347384) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2347384) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2347384) 'n_gpus_per_node': 8, (TaskRunner pid=2347384) 'nnodes': 1, (TaskRunner pid=2347384) 'project_name': 'SDPO-root', (TaskRunner pid=2347384) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2347384) 'resume_from_path': None, (TaskRunner pid=2347384) 'resume_mode': 'auto', (TaskRunner pid=2347384) 'rollout_data_dir': None, (TaskRunner pid=2347384) 'save_freq': 10, (TaskRunner pid=2347384) 'test_freq': 10, (TaskRunner pid=2347384) 'total_epochs': 30, (TaskRunner pid=2347384) 'total_training_steps': 100, (TaskRunner pid=2347384) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2347384) 'val_before_train': False, (TaskRunner pid=2347384) 'val_only': False, (TaskRunner pid=2347384) 'validation_data_dir': None}, (TaskRunner pid=2347384) 'transfer_queue': {'enable': False}, (TaskRunner pid=2347384) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2347384) 'dir': '/users/root/SDPO', (TaskRunner pid=2347384) 'log_dir': '/users/root/output', (TaskRunner pid=2347384) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2347384) use_critic=need_critic(config), (TaskRunner pid=2347384) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2347384) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2347384) Using dataset class: RLHFDataset (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2347384) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2347384) dataset len: 450 (TaskRunner pid=2347384) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2347384) WARNING:2026-07-03 07:28:14,003:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2347384) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2347384) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method wake_up to class .WorkerDict'> (WorkerDict pid=2347763) [W703 07:28:23.527944856 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:44381 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2347763) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2347766) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2347766) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2347764) [W703 07:28:23.702362713 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:44381 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2347763) Model config after override: Qwen3Config { (WorkerDict pid=2347763) "architectures": [ (WorkerDict pid=2347763) "Qwen3ForCausalLM" (WorkerDict pid=2347763) ], (WorkerDict pid=2347763) "attention_bias": false, (WorkerDict pid=2347763) "attention_dropout": 0.0, (WorkerDict pid=2347763) "dtype": "bfloat16", (WorkerDict pid=2347763) "eos_token_id": 151645, (WorkerDict pid=2347763) "head_dim": 128, (WorkerDict pid=2347763) "hidden_act": "silu", (WorkerDict pid=2347763) "hidden_size": 4096, (WorkerDict pid=2347763) "initializer_range": 0.02, (WorkerDict pid=2347763) "intermediate_size": 12288, (WorkerDict pid=2347763) "layer_types": [ (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention" (WorkerDict pid=2347763) ], (WorkerDict pid=2347763) "max_position_embeddings": 40960, (WorkerDict pid=2347763) "max_window_layers": 36, (WorkerDict pid=2347763) "model_type": "qwen3", (WorkerDict pid=2347763) "num_attention_heads": 32, (WorkerDict pid=2347763) "num_hidden_layers": 36, (WorkerDict pid=2347763) "num_key_value_heads": 8, (WorkerDict pid=2347763) "pad_token_id": 151643, (WorkerDict pid=2347763) "rms_norm_eps": 1e-06, (WorkerDict pid=2347763) "rope_scaling": null, (WorkerDict pid=2347763) "rope_theta": 1000000, (WorkerDict pid=2347763) "sliding_window": null, (WorkerDict pid=2347763) "tie_word_embeddings": false, (WorkerDict pid=2347763) "transformers_version": "4.57.1", (WorkerDict pid=2347763) "use_cache": true, (WorkerDict pid=2347763) "use_sliding_window": false, (WorkerDict pid=2347763) "vocab_size": 151936 (WorkerDict pid=2347763) } (WorkerDict pid=2347763) (WorkerDict pid=2347766) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2347766) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2347766) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2347769) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2347763) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2347764) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2347764) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2347770) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.95s/it] [repeated 15x across cluster] (WorkerDict pid=2347770) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.24s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.65s/it] [repeated 7x across cluster] (WorkerDict pid=2347763) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2347763) Actor use_remove_padding=True (WorkerDict pid=2347763) Actor use_fused_kernels=False (WorkerDict pid=2347763) Actor use_prefix_grouper=False (WorkerDict pid=2347770) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2347770) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2347766) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2347766) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2347764) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2347763) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2347763) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2347763) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2347763) warnings.warn( (WorkerDict pid=2347763) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2347763) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2347384) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2347769) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2347769) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2348653) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2348653) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2348653) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2348653) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2348652) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2348652) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2348652) ['serve', (vLLMHttpServer pid=2348652) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2348652) '--dtype', (vLLMHttpServer pid=2348652) 'bfloat16', (vLLMHttpServer pid=2348652) '--load_format', (vLLMHttpServer pid=2348652) 'dummy', (vLLMHttpServer pid=2348652) '--trust_remote_code', (vLLMHttpServer pid=2348652) '--max_model_len', (vLLMHttpServer pid=2348652) '40960', (vLLMHttpServer pid=2348652) '--max_num_seqs', (vLLMHttpServer pid=2348652) '1024', (vLLMHttpServer pid=2348652) '--enable_chunked_prefill', (vLLMHttpServer pid=2348652) '--max_num_batched_tokens', (vLLMHttpServer pid=2348652) '10240', (vLLMHttpServer pid=2348652) '--enable_prefix_caching', (vLLMHttpServer pid=2348652) '--enable_sleep_mode', (vLLMHttpServer pid=2348652) '--logprobs_mode', (vLLMHttpServer pid=2348652) 'processed_logprobs', (vLLMHttpServer pid=2348652) '--disable_custom_all_reduce', (vLLMHttpServer pid=2348652) '--gpu_memory_utilization', (vLLMHttpServer pid=2348652) '0.8', (vLLMHttpServer pid=2348652) '--disable_log_stats', (vLLMHttpServer pid=2348652) '--tensor_parallel_size', (vLLMHttpServer pid=2348652) '2', (vLLMHttpServer pid=2348652) '--seed', (vLLMHttpServer pid=2348652) '0', (vLLMHttpServer pid=2348652) '--override_generation_config', (vLLMHttpServer pid=2348652) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2348652) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2348652) '--hf_overrides', (vLLMHttpServer pid=2348652) '{}', (vLLMHttpServer pid=2348652) '--scheduling_policy', (vLLMHttpServer pid=2348652) 'fcfs'] (WorkerDict pid=2347769) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2348653) WARNING 07-03 07:29:13 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2347766) WARNING 07-03 07:29:21 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2348654) WARNING 07-03 07:29:14 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2347767) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2347763) 2026-07-03 07:29:35,888 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2347763) 2026-07-03 07:29:35,906 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2348654) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2348654) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (vLLMHttpServer pid=2348654) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2348654) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (WorkerDict pid=2347763) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) The folding stability score of a protein sequence is typically calculated using computational methods that evaluate the likelihood of the sequence folding into a stable three-dimensional structure. These methods often involve analyzing the energy landscape of the protein, predicting secondary structure elements, and assessing the overall stability based on amino acid interactions and hydrophobicity. (TaskRunner pid=2347384) (TaskRunner pid=2347384) In the absence of direct computation or access to a specific algorithm to calculate the folding stability score for the given protein sequence, we cannot determine the exact value. However, based on the provided options, we can infer that the correct answer is the one that represents a reasonable folding stability score. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Among the options, option B (1.69) and option C (2.49) are both positive values, which could indicate a relatively stable folding structure. Option A (-0.01) is very close to zero, which might suggest a slightly unstable or neutral folding stability. Option D (0.45) is also a positive value but smaller than B and C. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given that folding stability scores are typically reported as positive values, and considering the context of the question, the most plausible answer is B (1.69), as it represents a moderate to stable folding stability score. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) B (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 1.0 (TaskRunner pid=2347384) [acc] 1.0 (TaskRunner pid=2347384) [pred] B (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2347384) step:10 - global_seqlen/min:14026 - global_seqlen/max:26212 - global_seqlen/minmax_diff:12186 - global_seqlen/balanced_min:18467 - global_seqlen/balanced_max:25366 - global_seqlen/mean:19332.375 - actor/entropy:0.45262259244918823 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830507278442383 - training/rollout_probs_diff_mean:0.004972450900822878 - training/rollout_probs_diff_std:0.011055339127779007 - training/rollout_actor_probs_pearson_corr:0.9992586374282837 - rollout_corr/rollout_is_mean:0.9999414682388306 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1841186278616078e-05 - rollout_corr/rollout_is_max:1.7506085634231567 - rollout_corr/rollout_is_min:0.3154762089252472 - rollout_corr/rollout_is_std:0.03809688985347748 - rollout_corr/rollout_is_eff_sample_size:0.9985504926027746 - rollout_corr/rollout_is_seq_mean:0.999889612197876 - rollout_corr/rollout_is_seq_std:0.0027533581014722586 - rollout_corr/rollout_is_seq_max:1.010258674621582 - rollout_corr/rollout_is_seq_min:0.9903730750083923 - rollout_corr/rollout_is_seq_max_deviation:0.010258674621582031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.716814047191292) - rollout_corr/training_log_ppl:np.float64(0.5270733479010232) - rollout_corr/kl:np.float64(0.0010048211948259222) - rollout_corr/k3_kl:np.float64(0.0008655035604760997) - rollout_corr/rollout_ppl:np.float64(1.715072752442211) - rollout_corr/rollout_log_ppl:np.float64(0.5260685242756153) - rollout_corr/log_ppl_diff:np.float64(0.0010048236254078802) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021937599194643553) - rollout_corr/log_ppl_diff_max:np.float64(0.00876617431640625) - rollout_corr/log_ppl_diff_min:np.float64(-0.009220004081726074) - rollout_corr/ppl_ratio:np.float64(1.0010089005809277) - rollout_corr/chi2_token:np.float64(0.0014747201930731535) - rollout_corr/chi2_seq:np.float64(0.7448661010712385) - actor/pg_loss:np.float64(-0.00027040555141866207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006960646828702011) - actor/ppo_kl:np.float64(3.835439729282797e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5604352355003357) - perf/mfu/actor:np.float64(0.07364472637593265) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(98.74493789672852) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.305) - val-aux/sciknoweval/reward/std@16:np.float64(0.34964298228177393) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.45586) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.34684612554072713) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.1565) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24691334720794672) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.30598000000000003) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.350415368201675) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6034799999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2922414405058621) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.0688) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13262228575634746) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.30114) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.3016085903429573) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7244600000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.20789043475164487) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.025199999999999997) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.057717604537027185) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.29326) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.2483461544296699) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8033999999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.11852761284640682) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.0065000000000000014) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.023749015715872786) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.28244) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1927622182845085) - val-aux/sciknoweval/score/mean@16:np.float64(0.305) - val-aux/sciknoweval/score/std@16:np.float64(0.34964298228177393) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.45586) - val-aux/sciknoweval/score/best@2/std:np.float64(0.34684612554072713) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.1565) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24691334720794672) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.30598000000000003) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.350415368201675) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6034799999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2922414405058621) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.0688) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13262228575634746) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.30114) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.3016085903429573) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7244600000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.20789043475164487) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.025199999999999997) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.057717604537027185) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.29326) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.2483461544296699) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8033999999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.11852761284640682) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.0065000000000000014) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.023749015715872786) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.28244) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1927622182845085) - val-core/sciknoweval/acc/mean@16:np.float64(0.305) - val-aux/sciknoweval/acc/std@16:np.float64(0.34964298228177393) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.45586) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.34684612554072713) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.1565) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24691334720794672) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.30598000000000003) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.350415368201675) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6034799999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2922414405058621) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.0688) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13262228575634746) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.30114) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.3016085903429573) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7244600000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.20789043475164487) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.025199999999999997) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.057717604537027185) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.29326) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.2483461544296699) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8033999999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.11852761284640682) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.0065000000000000014) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.023749015715872786) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.28244) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1927622182845085) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.97125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.060544496479082686) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9928400000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02934722149080065) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9498399999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07393366420051682) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9717999999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06020345104042302) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99938) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.005582341464791209) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.91966) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07936022015675398) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.98834) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03981096521967535) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.88452) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07223829094842676) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9972) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.019694784702167224) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8517600000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05263531516347757) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004406536886723831) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.32421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.32421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.029959680512547493 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.029959680512547493 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:329.88671875 - response_length/max:8192.0 - response_length/min:128.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:329.88671875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019344687461853027 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5390802696347237) - timing_s/agent_loop/generate_sequences/max:np.float64(56.44360770471394) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5275733740854776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(56.44360770471394) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:62.94056263193488 - timing_s/reward:0.05771280638873577 - timing_s/old_log_prob:3.3531495556235313 - timing_s/adv:0.13802666775882244 - timing_s/update_actor:13.871477348729968 - timing_s/step:80.44779045693576 - timing_s/testing:96.19575156643987 - timing_s/save_checkpoint:12.671725941821933 - timing_s/stop_profile:0.00013373419642448425 - timing_per_token_ms/update_actor:0.08969072183791417 - timing_per_token_ms/gen:0.7452909098996445 - timing_per_token_ms/adv:0.0008924580383865307 - perf/total_num_tokens:154659 - perf/time_per_step:80.44779045693576 - perf/throughput:240.30958327374756 (TaskRunner pid=2347384) Training Progress: 10%|█ | 10/100 [10:23<2:12:15, 88.17s/it] (TaskRunner pid=2347384) step:11 - global_seqlen/min:16957 - global_seqlen/max:30433 - global_seqlen/minmax_diff:13476 - global_seqlen/balanced_min:22590 - global_seqlen/balanced_max:25365 - global_seqlen/mean:22965.625 - actor/entropy:0.4155322313308716 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3120696544647217 - training/rollout_probs_diff_mean:0.004567084833979607 - training/rollout_probs_diff_std:0.010465013794600964 - training/rollout_actor_probs_pearson_corr:0.9992883801460266 - rollout_corr/rollout_is_mean:0.9998583197593689 - rollout_corr/rollout_is_ratio_fraction_high:1.0084203495352995e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.016840699070599e-05 - rollout_corr/rollout_is_max:2.0753023624420166 - rollout_corr/rollout_is_min:0.4422110319137573 - rollout_corr/rollout_is_std:0.0358213447034359 - rollout_corr/rollout_is_eff_sample_size:0.9987180000038098 - rollout_corr/rollout_is_seq_mean:0.9998087882995605 - rollout_corr/rollout_is_seq_std:0.002214111154899001 - rollout_corr/rollout_is_seq_max:1.0086524486541748 - rollout_corr/rollout_is_seq_min:0.993055522441864 - rollout_corr/rollout_is_seq_max_deviation:0.008652448654174805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6284825187176466) - rollout_corr/training_log_ppl:np.float64(0.47535870561841875) - rollout_corr/kl:np.float64(0.0009332117763483083) - rollout_corr/k3_kl:np.float64(0.0008248986909222822) - rollout_corr/rollout_ppl:np.float64(1.6268996740691364) - rollout_corr/rollout_log_ppl:np.float64(0.47442549155675806) - rollout_corr/log_ppl_diff:np.float64(0.0009332140616606921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019218551169615239) - rollout_corr/log_ppl_diff_max:np.float64(0.007362186908721924) - rollout_corr/log_ppl_diff_min:np.float64(-0.007423877716064453) - rollout_corr/ppl_ratio:np.float64(1.0009363759309053) - rollout_corr/chi2_token:np.float64(0.001480589620769024) - rollout_corr/chi2_seq:np.float64(0.6636504703201354) - actor/pg_loss:np.float64(0.0001596701331436634) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046512034583656714) - actor/ppo_kl:np.float64(-2.2117470098947933e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5356682911515236) - perf/mfu/actor:np.float64(0.08978195743431006) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(102.40783309936523) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004459738731384277 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004459738731384277 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:387.36328125 - response_length/max:4649.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:387.36328125 - response_length_non_aborted/max:4649.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001032371073961258 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7794717792421579) - timing_s/agent_loop/generate_sequences/max:np.float64(28.687389539554715) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.100075885755359) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.687389539554715) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:433 - timing_s/agent_loop/slowest/response_length:4649 - timing_s/gen:35.13612602651119 - timing_s/reward:0.05825859680771828 - timing_s/old_log_prob:3.552225437015295 - timing_s/adv:0.17767742276191711 - timing_s/update_actor:13.233200673013926 - timing_s/step:52.25685699656606 - timing_s/stop_profile:0.00011438876390457153 - timing_per_token_ms/update_actor:0.07202721825017785 - timing_per_token_ms/gen:0.3543198308527322 - timing_per_token_ms/adv:0.0009670835366004469 - perf/total_num_tokens:183725 - perf/time_per_step:52.25685699656606 - perf/throughput:439.4758184846275 (TaskRunner pid=2347384) Training Progress: 11%|█ | 11/100 [11:16<1:54:30, 77.20s/it] (TaskRunner pid=2347384) step:12 - global_seqlen/min:18215 - global_seqlen/max:29458 - global_seqlen/minmax_diff:11243 - global_seqlen/balanced_min:23841 - global_seqlen/balanced_max:27478 - global_seqlen/mean:24339.0 - actor/entropy:0.43766021728515625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30099600553512573 - training/rollout_probs_diff_mean:0.004334617871791124 - training/rollout_probs_diff_std:0.009989089332520962 - training/rollout_actor_probs_pearson_corr:0.9993811249732971 - rollout_corr/rollout_is_mean:1.000182867050171 - rollout_corr/rollout_is_ratio_fraction_high:9.775553735380527e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.887776594841853e-05 - rollout_corr/rollout_is_max:2.1124563217163086 - rollout_corr/rollout_is_min:0.011348603293299675 - rollout_corr/rollout_is_std:0.034875381737947464 - rollout_corr/rollout_is_eff_sample_size:0.9987856608682638 - rollout_corr/rollout_is_seq_mean:1.000220537185669 - rollout_corr/rollout_is_seq_std:0.0024740397930145264 - rollout_corr/rollout_is_seq_max:1.0120216608047485 - rollout_corr/rollout_is_seq_min:0.9940353035926819 - rollout_corr/rollout_is_seq_max_deviation:0.012021660804748535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6821262426674366) - rollout_corr/training_log_ppl:np.float64(0.5046567447425332) - rollout_corr/kl:np.float64(0.0005845932231274276) - rollout_corr/k3_kl:np.float64(0.0007912340478242186) - rollout_corr/rollout_ppl:np.float64(1.681106602307409) - rollout_corr/rollout_log_ppl:np.float64(0.5040721487021074) - rollout_corr/log_ppl_diff:np.float64(0.0005845960404258221) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019499089976307005) - rollout_corr/log_ppl_diff_max:np.float64(0.013257354497909546) - rollout_corr/log_ppl_diff_min:np.float64(-0.010273128747940063) - rollout_corr/ppl_ratio:np.float64(1.000588194001466) - rollout_corr/chi2_token:np.float64(0.001965596806257963) - rollout_corr/chi2_seq:np.float64(1.858226215466857) - actor/pg_loss:np.float64(0.0003983316710218787) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006193824146976112) - actor/ppo_kl:np.float64(6.872035001781462e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5202421024441719) - perf/mfu/actor:np.float64(0.09425964715479349) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(101.28682327270508) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013849515467882156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013849515467882156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:399.59375 - response_length/max:5387.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:399.59375 - response_length_non_aborted/max:5387.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.916350245475769e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.688298499211669) - timing_s/agent_loop/generate_sequences/max:np.float64(32.550289833918214) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.273808764126443) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.550289833918214) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:5387 - timing_s/gen:39.82046948000789 - timing_s/reward:0.0908251591026783 - timing_s/old_log_prob:3.288546323776245 - timing_s/adv:0.13700509257614613 - timing_s/update_actor:13.3698172532022 - timing_s/step:56.79229616187513 - timing_s/stop_profile:0.0001241397112607956 - timing_per_token_ms/update_actor:0.06866457770040983 - timing_per_token_ms/gen:0.3892671216861645 - timing_per_token_ms/adv:0.0007036294248744101 - perf/total_num_tokens:194712 - perf/time_per_step:56.79229616187513 - perf/throughput:428.5616473513684 (TaskRunner pid=2347384) Training Progress: 12%|█▏ | 12/100 [12:12<1:44:08, 71.01s/it] (TaskRunner pid=2347384) step:13 - global_seqlen/min:16426 - global_seqlen/max:28676 - global_seqlen/minmax_diff:12250 - global_seqlen/balanced_min:20919 - global_seqlen/balanced_max:27689 - global_seqlen/mean:21768.75 - actor/entropy:0.45051950216293335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25071096420288086 - training/rollout_probs_diff_mean:0.004617288708686829 - training/rollout_probs_diff_std:0.010263320989906788 - training/rollout_actor_probs_pearson_corr:0.999343752861023 - rollout_corr/rollout_is_mean:0.9999645948410034 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.044822874973761e-05 - rollout_corr/rollout_is_max:1.7554703950881958 - rollout_corr/rollout_is_min:0.37157756090164185 - rollout_corr/rollout_is_std:0.03532538563013077 - rollout_corr/rollout_is_eff_sample_size:0.9987534346031791 - rollout_corr/rollout_is_seq_mean:0.999995231628418 - rollout_corr/rollout_is_seq_std:0.002299003768712282 - rollout_corr/rollout_is_seq_max:1.0069160461425781 - rollout_corr/rollout_is_seq_min:0.9921134114265442 - rollout_corr/rollout_is_seq_max_deviation:0.00788658857345581 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6914544617757201) - rollout_corr/training_log_ppl:np.float64(0.5109681343019474) - rollout_corr/kl:np.float64(0.0007979853619914934) - rollout_corr/k3_kl:np.float64(0.00077781562168866) - rollout_corr/rollout_ppl:np.float64(1.6901205792091787) - rollout_corr/rollout_log_ppl:np.float64(0.5101701469830004) - rollout_corr/log_ppl_diff:np.float64(0.0007979873189469799) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018314038024982437) - rollout_corr/log_ppl_diff_max:np.float64(0.008499443531036377) - rollout_corr/log_ppl_diff_min:np.float64(-0.005160987377166748) - rollout_corr/ppl_ratio:np.float64(1.0008007904980332) - rollout_corr/chi2_token:np.float64(0.0015173715073615313) - rollout_corr/chi2_seq:np.float64(0.3772704761940986) - actor/pg_loss:np.float64(-0.0003307295264676213) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005128576804054319) - actor/ppo_kl:np.float64(6.100696820965368e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.45558615028858185) - perf/mfu/actor:np.float64(0.0824730901618031) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(101.28923034667969) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06721868366003036 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06721868366003036 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:373.8671875 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:373.8671875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.137524127960205e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.67017375305295) - timing_s/agent_loop/generate_sequences/max:np.float64(58.25485855713487) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.205507205348113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.25485855713487) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.14460217580199 - timing_s/reward:0.06125919334590435 - timing_s/old_log_prob:3.345760280266404 - timing_s/adv:0.16389351710677147 - timing_s/update_actor:13.742141423746943 - timing_s/step:82.54458156228065 - timing_s/stop_profile:0.0001163724809885025 - timing_per_token_ms/update_actor:0.07890979858597154 - timing_per_token_ms/gen:0.6806457232870337 - timing_per_token_ms/adv:0.0009411054671649237 - perf/total_num_tokens:174150 - perf/time_per_step:82.54458156228065 - perf/throughput:263.72112606295394 (TaskRunner pid=2347384) Training Progress: 13%|█▎ | 13/100 [13:35<1:48:02, 74.52s/it] (TaskRunner pid=2347384) step:14 - global_seqlen/min:14671 - global_seqlen/max:39208 - global_seqlen/minmax_diff:24537 - global_seqlen/balanced_min:21329 - global_seqlen/balanced_max:27996 - global_seqlen/mean:22634.625 - actor/entropy:0.4110657870769501 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016807436943054 - training/rollout_probs_diff_mean:0.00429363502189517 - training/rollout_probs_diff_std:0.010314399376511574 - training/rollout_actor_probs_pearson_corr:0.9993447661399841 - rollout_corr/rollout_is_mean:0.9998406171798706 - rollout_corr/rollout_is_ratio_fraction_high:1.9119544958812185e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8679316528723575e-05 - rollout_corr/rollout_is_max:2.719468116760254 - rollout_corr/rollout_is_min:0.39786025881767273 - rollout_corr/rollout_is_std:0.03521892428398132 - rollout_corr/rollout_is_eff_sample_size:0.9987608072183694 - rollout_corr/rollout_is_seq_mean:0.9998125433921814 - rollout_corr/rollout_is_seq_std:0.0023194169625639915 - rollout_corr/rollout_is_seq_max:1.006919264793396 - rollout_corr/rollout_is_seq_min:0.9936529397964478 - rollout_corr/rollout_is_seq_max_deviation:0.006919264793395996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.67210001219064) - rollout_corr/training_log_ppl:np.float64(0.4983691323141102) - rollout_corr/kl:np.float64(0.0009185323109477395) - rollout_corr/k3_kl:np.float64(0.0007933990461879148) - rollout_corr/rollout_ppl:np.float64(1.6704831109382212) - rollout_corr/rollout_log_ppl:np.float64(0.49745059617998777) - rollout_corr/log_ppl_diff:np.float64(0.0009185361341224052) - rollout_corr/log_ppl_abs_diff:np.float64(0.001841111145040486) - rollout_corr/log_ppl_diff_max:np.float64(0.008407264947891235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005116939544677734) - rollout_corr/ppl_ratio:np.float64(1.0009215117897838) - rollout_corr/chi2_token:np.float64(0.0013447413221001625) - rollout_corr/chi2_seq:np.float64(0.33762073097750545) - actor/pg_loss:np.float64(0.00017369771376252174) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007912135890819627) - actor/ppo_kl:np.float64(-9.891378601523115e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4692867696285248) - perf/mfu/actor:np.float64(0.08400188820824843) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(101.32085037231445) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03370058909058571 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.03370058909058571 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:408.61328125 - response_length/max:8192.0 - response_length/min:119.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:408.61328125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010697729885578156 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5121872909367085) - timing_s/agent_loop/generate_sequences/max:np.float64(58.620669377967715) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.286885734763928) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.620669377967715) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.46320316195488 - timing_s/reward:0.061744675040245056 - timing_s/old_log_prob:3.311249475926161 - timing_s/adv:0.13611549139022827 - timing_s/update_actor:14.122505748644471 - timing_s/step:83.18165424838662 - timing_s/stop_profile:0.00011662021279335022 - timing_per_token_ms/update_actor:0.0779917148431025 - timing_per_token_ms/gen:0.6258133278710853 - timing_per_token_ms/adv:0.0007516995056811648 - perf/total_num_tokens:181077 - perf/time_per_step:83.18165424838662 - perf/throughput:272.1107821733302 (TaskRunner pid=2347384) Training Progress: 14%|█▍ | 14/100 [14:58<1:50:34, 77.15s/it] (TaskRunner pid=2347384) step:15 - global_seqlen/min:16770 - global_seqlen/max:25955 - global_seqlen/minmax_diff:9185 - global_seqlen/balanced_min:19227 - global_seqlen/balanced_max:19234 - global_seqlen/mean:19231.875 - actor/entropy:0.49017786979675293 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772047519683838 - training/rollout_probs_diff_mean:0.005305732134729624 - training/rollout_probs_diff_std:0.011075073853135109 - training/rollout_actor_probs_pearson_corr:0.9992448687553406 - rollout_corr/rollout_is_mean:0.9997841715812683 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2659669664571993e-05 - rollout_corr/rollout_is_max:1.859199047088623 - rollout_corr/rollout_is_min:0.44539427757263184 - rollout_corr/rollout_is_std:0.03849146142601967 - rollout_corr/rollout_is_eff_sample_size:0.9985199455019984 - rollout_corr/rollout_is_seq_mean:0.9997751116752625 - rollout_corr/rollout_is_seq_std:0.0026260388549417257 - rollout_corr/rollout_is_seq_max:1.008506417274475 - rollout_corr/rollout_is_seq_min:0.9886805415153503 - rollout_corr/rollout_is_seq_max_deviation:0.011319458484649658 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6824355935677886) - rollout_corr/training_log_ppl:np.float64(0.5090422219946049) - rollout_corr/kl:np.float64(0.001040822978014333) - rollout_corr/k3_kl:np.float64(0.0008659341230554674) - rollout_corr/rollout_ppl:np.float64(1.680602803826332) - rollout_corr/rollout_log_ppl:np.float64(0.5080013968981802) - rollout_corr/log_ppl_diff:np.float64(0.001040825096424669) - rollout_corr/log_ppl_abs_diff:np.float64(0.002198683505412191) - rollout_corr/log_ppl_diff_max:np.float64(0.009897172451019287) - rollout_corr/log_ppl_diff_min:np.float64(-0.007486939430236816) - rollout_corr/ppl_ratio:np.float64(1.001044871052727) - rollout_corr/chi2_token:np.float64(0.0013809786178171635) - rollout_corr/chi2_seq:np.float64(0.2935559705365449) - actor/pg_loss:np.float64(-0.00017933628987520933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008264910120487912) - actor/ppo_kl:np.float64(8.757520348012804e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.48963937163352966) - perf/mfu/actor:np.float64(0.07772259665729145) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.28882598876953) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001990733202546835 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001990733202546835 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:308.55859375 - response_length/max:980.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.55859375 - response_length_non_aborted/max:980.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00030867382884025574 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3679221644997597) - timing_s/agent_loop/generate_sequences/max:np.float64(7.741090871393681) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4445935255935183) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.741090871393681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:980 - timing_s/gen:14.896692911162972 - timing_s/reward:0.05144878849387169 - timing_s/old_log_prob:3.2346764598041773 - timing_s/adv:0.14543371461331844 - timing_s/update_actor:12.904560782015324 - timing_s/step:31.340551301836967 - timing_s/stop_profile:0.00012101233005523682 - timing_per_token_ms/update_actor:0.08387482228081845 - timing_per_token_ms/gen:0.1885872176724307 - timing_per_token_ms/adv:0.0009452647922610148 - perf/total_num_tokens:153855 - perf/time_per_step:31.340551301836967 - perf/throughput:613.6418857083972 (TaskRunner pid=2347384) Training Progress: 15%|█▌ | 15/100 [15:30<1:29:52, 63.44s/it] (TaskRunner pid=2347384) step:16 - global_seqlen/min:18245 - global_seqlen/max:24988 - global_seqlen/minmax_diff:6743 - global_seqlen/balanced_min:21582 - global_seqlen/balanced_max:21592 - global_seqlen/mean:21588.375 - actor/entropy:0.46256548166275024 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2309231162071228 - training/rollout_probs_diff_mean:0.005185081623494625 - training/rollout_probs_diff_std:0.011045632883906364 - training/rollout_actor_probs_pearson_corr:0.9992262721061707 - rollout_corr/rollout_is_mean:0.9999391436576843 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.128833810071228e-05 - rollout_corr/rollout_is_max:1.7464262247085571 - rollout_corr/rollout_is_min:0.4746478796005249 - rollout_corr/rollout_is_std:0.03808280825614929 - rollout_corr/rollout_is_eff_sample_size:0.9985516812436948 - rollout_corr/rollout_is_seq_mean:0.9998874068260193 - rollout_corr/rollout_is_seq_std:0.002428104868158698 - rollout_corr/rollout_is_seq_max:1.01023268699646 - rollout_corr/rollout_is_seq_min:0.9897691607475281 - rollout_corr/rollout_is_seq_max_deviation:0.010232686996459961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.641688687261194) - rollout_corr/training_log_ppl:np.float64(0.4859840482240543) - rollout_corr/kl:np.float64(0.000994573286405398) - rollout_corr/k3_kl:np.float64(0.0008416437668756771) - rollout_corr/rollout_ppl:np.float64(1.6399632166139781) - rollout_corr/rollout_log_ppl:np.float64(0.4849894735380076) - rollout_corr/log_ppl_diff:np.float64(0.0009945746860466897) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020759932813234627) - rollout_corr/log_ppl_diff_max:np.float64(0.008678853511810303) - rollout_corr/log_ppl_diff_min:np.float64(-0.009289979934692383) - rollout_corr/ppl_ratio:np.float64(1.0009982739575207) - rollout_corr/chi2_token:np.float64(0.001368396682664752) - rollout_corr/chi2_seq:np.float64(0.8329015655908734) - actor/pg_loss:np.float64(-0.0002698972821235657) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005803697106330219) - actor/ppo_kl:np.float64(8.798259023556909e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5201293975114822) - perf/mfu/actor:np.float64(0.08792344894975813) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.23332977294922) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021574837155640125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021574837155640125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.04296875 - response_length/max:1053.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.04296875 - response_length_non_aborted/max:1053.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001698676496744156 - timing_s/agent_loop/generate_sequences/min:np.float64(1.689747143536806) - timing_s/agent_loop/generate_sequences/max:np.float64(8.236624287441373) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8304059656438767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.236624287441373) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1053 - timing_s/gen:16.13315940834582 - timing_s/reward:0.06064635515213013 - timing_s/old_log_prob:3.0629591830074787 - timing_s/adv:0.17271914333105087 - timing_s/update_actor:12.850557547062635 - timing_s/step:32.3675423245877 - timing_s/stop_profile:0.00011733546853065491 - timing_per_token_ms/update_actor:0.07440669774278191 - timing_per_token_ms/gen:0.1821165566995814 - timing_per_token_ms/adv:0.0010000703117479365 - perf/total_num_tokens:172707 - perf/time_per_step:32.3675423245877 - perf/throughput:666.9760336916464 (TaskRunner pid=2347384) Training Progress: 16%|█▌ | 16/100 [16:02<1:15:44, 54.11s/it] (TaskRunner pid=2347384) step:17 - global_seqlen/min:14579 - global_seqlen/max:23993 - global_seqlen/minmax_diff:9414 - global_seqlen/balanced_min:20135 - global_seqlen/balanced_max:20148 - global_seqlen/mean:20144.625 - actor/entropy:0.47733035683631897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2896254360675812 - training/rollout_probs_diff_mean:0.005261630285531282 - training/rollout_probs_diff_std:0.011144670657813549 - training/rollout_actor_probs_pearson_corr:0.9992367029190063 - rollout_corr/rollout_is_mean:0.9999669790267944 - rollout_corr/rollout_is_ratio_fraction_high:1.1454622836026829e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1454622836026829e-05 - rollout_corr/rollout_is_max:2.0969088077545166 - rollout_corr/rollout_is_min:0.48796799778938293 - rollout_corr/rollout_is_std:0.03874843195080757 - rollout_corr/rollout_is_eff_sample_size:0.9985005722393395 - rollout_corr/rollout_is_seq_mean:0.9999296069145203 - rollout_corr/rollout_is_seq_std:0.0025035692378878593 - rollout_corr/rollout_is_seq_max:1.0074434280395508 - rollout_corr/rollout_is_seq_min:0.989840030670166 - rollout_corr/rollout_is_seq_max_deviation:0.010159969329833984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.689788539428264) - rollout_corr/training_log_ppl:np.float64(0.5136786702787504) - rollout_corr/kl:np.float64(0.0010801191343556837) - rollout_corr/k3_kl:np.float64(0.0009097063799572425) - rollout_corr/rollout_ppl:np.float64(1.6878817342221737) - rollout_corr/rollout_log_ppl:np.float64(0.5125985508784652) - rollout_corr/log_ppl_diff:np.float64(0.0010801194002851844) - rollout_corr/log_ppl_abs_diff:np.float64(0.002021018648520112) - rollout_corr/log_ppl_diff_max:np.float64(0.009488582611083984) - rollout_corr/log_ppl_diff_min:np.float64(-0.006097257137298584) - rollout_corr/ppl_ratio:np.float64(1.001083607552573) - rollout_corr/chi2_token:np.float64(0.0014436279889196157) - rollout_corr/chi2_seq:np.float64(0.4424182176589966) - actor/pg_loss:np.float64(-1.9055441953241825e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010638892772476538) - actor/ppo_kl:np.float64(0.00019094234705008262) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4444914981722832) - perf/mfu/actor:np.float64(0.08021553795910458) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.20829391479492) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.23046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.23046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00047966232523322105 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00047966232523322105 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.01953125 - response_length/max:1183.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.01953125 - response_length_non_aborted/max:1183.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010183639824390411 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5702809430658817) - timing_s/agent_loop/generate_sequences/max:np.float64(9.146460255607963) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.727873096773692) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.146460255607963) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1183 - timing_s/gen:15.822298798710108 - timing_s/reward:0.05862102285027504 - timing_s/old_log_prob:3.125773211941123 - timing_s/adv:0.14922525733709335 - timing_s/update_actor:13.030285563319921 - timing_s/step:32.271964421495795 - timing_s/stop_profile:0.00011430494487285614 - timing_per_token_ms/update_actor:0.08085460490900129 - timing_per_token_ms/gen:0.18123846002577412 - timing_per_token_ms/adv:0.0009259619956756042 - perf/total_num_tokens:161157 - perf/time_per_step:32.271964421495795 - perf/throughput:624.2144028450284 (TaskRunner pid=2347384) Training Progress: 17%|█▋ | 17/100 [16:35<1:05:47, 47.56s/it] (TaskRunner pid=2347384) step:18 - global_seqlen/min:14050 - global_seqlen/max:31007 - global_seqlen/minmax_diff:16957 - global_seqlen/balanced_min:21074 - global_seqlen/balanced_max:27792 - global_seqlen/mean:21943.25 - actor/entropy:0.41611501574516296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30242347717285156 - training/rollout_probs_diff_mean:0.005012628622353077 - training/rollout_probs_diff_std:0.011453614570200443 - training/rollout_actor_probs_pearson_corr:0.999149739742279 - rollout_corr/rollout_is_mean:1.0000622272491455 - rollout_corr/rollout_is_ratio_fraction_high:1.0493399713595863e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0493399713595863e-05 - rollout_corr/rollout_is_max:2.399968147277832 - rollout_corr/rollout_is_min:0.20653177797794342 - rollout_corr/rollout_is_std:0.038318414241075516 - rollout_corr/rollout_is_eff_sample_size:0.9985340896470543 - rollout_corr/rollout_is_seq_mean:1.0000394582748413 - rollout_corr/rollout_is_seq_std:0.002293739467859268 - rollout_corr/rollout_is_seq_max:1.0059384107589722 - rollout_corr/rollout_is_seq_min:0.9915452599525452 - rollout_corr/rollout_is_seq_max_deviation:0.008454740047454834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6364765437319875) - rollout_corr/training_log_ppl:np.float64(0.48095705287414603) - rollout_corr/kl:np.float64(0.0008645531091753256) - rollout_corr/k3_kl:np.float64(0.000911699380878872) - rollout_corr/rollout_ppl:np.float64(1.6350416275672615) - rollout_corr/rollout_log_ppl:np.float64(0.4800924996234244) - rollout_corr/log_ppl_diff:np.float64(0.0008645532507216558) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019391910027479753) - rollout_corr/log_ppl_diff_max:np.float64(0.011366486549377441) - rollout_corr/log_ppl_diff_min:np.float64(-0.005639404058456421) - rollout_corr/ppl_ratio:np.float64(1.0008679183665663) - rollout_corr/chi2_token:np.float64(0.0019431104883551598) - rollout_corr/chi2_seq:np.float64(0.5791504352819175) - actor/pg_loss:np.float64(0.00032479327637702227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010526327953357395) - actor/ppo_kl:np.float64(4.073885254030074e-05) - actor/pg_clipfrac_lower:np.float64(6.4566115725028794e-06) - actor/grad_norm:np.float64(0.48805930465459824) - perf/mfu/actor:np.float64(0.08156170908553631) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.15203094482422) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04970329999923706 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04970329999923706 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.2578125 - response_length/max:8192.0 - response_length/min:103.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:372.2578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018807873129844666 - timing_s/agent_loop/generate_sequences/min:np.float64(1.205347741022706) - timing_s/agent_loop/generate_sequences/max:np.float64(58.50628229603171) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.006803513904742) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.50628229603171) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.4495088905096 - timing_s/reward:0.059897202998399734 - timing_s/old_log_prob:3.325123270973563 - timing_s/adv:0.12080579437315464 - timing_s/update_actor:14.093188527971506 - timing_s/step:83.13446616567671 - timing_s/stop_profile:0.00012601912021636963 - timing_per_token_ms/update_actor:0.08028202595314907 - timing_per_token_ms/gen:0.6867878537903167 - timing_per_token_ms/adv:0.0006881717291943686 - perf/total_num_tokens:175546 - perf/time_per_step:83.13446616567671 - perf/throughput:263.9488892160541 (TaskRunner pid=2347384) Training Progress: 18%|█▊ | 18/100 [17:58<1:19:37, 58.26s/it] (TaskRunner pid=2347384) step:19 - global_seqlen/min:14821 - global_seqlen/max:32064 - global_seqlen/minmax_diff:17243 - global_seqlen/balanced_min:21027 - global_seqlen/balanced_max:25572 - global_seqlen/mean:21620.625 - actor/entropy:0.3882409930229187 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27794426679611206 - training/rollout_probs_diff_mean:0.004868786782026291 - training/rollout_probs_diff_std:0.011396627873182297 - training/rollout_actor_probs_pearson_corr:0.9991189241409302 - rollout_corr/rollout_is_mean:1.000097632408142 - rollout_corr/rollout_is_ratio_fraction_high:1.0341154847992584e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.035048723220825 - rollout_corr/rollout_is_min:0.5708956718444824 - rollout_corr/rollout_is_std:0.03782682865858078 - rollout_corr/rollout_is_eff_sample_size:0.9985715319653053 - rollout_corr/rollout_is_seq_mean:1.0002447366714478 - rollout_corr/rollout_is_seq_std:0.0023739675525575876 - rollout_corr/rollout_is_seq_max:1.0081961154937744 - rollout_corr/rollout_is_seq_min:0.9933203458786011 - rollout_corr/rollout_is_seq_max_deviation:0.008196115493774414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5852337004616857) - rollout_corr/training_log_ppl:np.float64(0.4482002856093459) - rollout_corr/kl:np.float64(0.0009470764403403109) - rollout_corr/k3_kl:np.float64(0.0009331986865674935) - rollout_corr/rollout_ppl:np.float64(1.5837227194570005) - rollout_corr/rollout_log_ppl:np.float64(0.4472532053478062) - rollout_corr/log_ppl_diff:np.float64(0.0009470802615396678) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020817756303586066) - rollout_corr/log_ppl_diff_max:np.float64(0.00791981816291809) - rollout_corr/log_ppl_diff_min:np.float64(-0.007333278656005859) - rollout_corr/ppl_ratio:np.float64(1.0009507073555142) - rollout_corr/chi2_token:np.float64(0.0018556120339781046) - rollout_corr/chi2_seq:np.float64(0.8564345519989729) - actor/pg_loss:np.float64(0.0005588408093899488) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011391409620955528) - actor/ppo_kl:np.float64(0.0003408312764214827) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5193510204553604) - perf/mfu/actor:np.float64(0.06523030261134072) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(137.5371389389038) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01663116179406643 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01663116179406643 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.73828125 - response_length/max:6077.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.73828125 - response_length_non_aborted/max:6077.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.726174175739288e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7443513739854097) - timing_s/agent_loop/generate_sequences/max:np.float64(36.85969475097954) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.116641358414199) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.85969475097954) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:6077 - timing_s/gen:44.36828749626875 - timing_s/reward:0.06108302250504494 - timing_s/old_log_prob:3.1765850484371185 - timing_s/adv:0.11546890065073967 - timing_s/update_actor:17.13972050882876 - timing_s/step:64.9494182318449 - timing_s/stop_profile:0.0001245979219675064 - timing_per_token_ms/update_actor:0.09909357678622126 - timing_per_token_ms/gen:0.45881932447719 - timing_per_token_ms/adv:0.000667585353399472 - perf/total_num_tokens:172965 - perf/time_per_step:64.9494182318449 - perf/throughput:332.88404405444453 (TaskRunner pid=2347384) Training Progress: 19%|█▉ | 19/100 [19:03<1:21:23, 60.28s/it]